2023-06-08T03:37:37.294 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd 2023-06-08T03:37:37.295 DEBUG:teuthology.run:Teuthology command: teuthology --name xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi --owner scheduled_xiubli@teuthology --archive /home/teuthworker/archive/xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7298112 --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} -- /home/teuthworker/archive/xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7298112/orig.config.yaml 2023-06-08T03:37:37.338 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-08T03:37:37.427 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7298112 branch: wip-lxb-fscrypt-20230607-0901 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: null first_in_suite: false job_id: '7298112' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-lxb-fscrypt-20230607-0901 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lxb-fscrypt-20230607-0901 sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b owner: scheduled_xiubli@teuthology priority: 110 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-lxb-fscrypt-20230607-0901 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi156.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61 timestamp: 2023-06-08_03:32:08 tube: smithi user: xiubli verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-08T03:37:37.427 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa; will attempt to use it 2023-06-08T03:37:37.428 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa/tasks 2023-06-08T03:37:37.428 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-08T03:37:37.430 INFO:teuthology.task.internal:Checking packages... 2023-06-08T03:37:37.459 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a02638f808b8fe11bccdd01222c0e6e2eb71120b' 2023-06-08T03:37:37.459 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-08T03:37:37.460 INFO:teuthology.packaging:ref: None 2023-06-08T03:37:37.460 INFO:teuthology.packaging:tag: None 2023-06-08T03:37:37.460 INFO:teuthology.packaging:branch: wip-lxb-fscrypt-20230607-0901 2023-06-08T03:37:37.461 INFO:teuthology.packaging:sha1: a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-08T03:37:37.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-lxb-fscrypt-20230607-0901 2023-06-08T03:37:37.678 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4313.ga02638f8 2023-06-08T03:37:37.678 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-08T03:37:37.684 INFO:teuthology.task.internal:no buildpackages task found 2023-06-08T03:37:37.685 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-08T03:37:37.708 INFO:teuthology.task.internal:Saving configuration 2023-06-08T03:37:37.725 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-08T03:37:37.736 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-08T03:37:37.765 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7298112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-08 03:32:16.358171', 'locked_by': 'scheduled_xiubli@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-08T03:37:37.784 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7298112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-08 03:32:16.356536', 'locked_by': 'scheduled_xiubli@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-08T03:37:37.785 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-08T03:37:37.791 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-08T03:37:37.791 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-08T03:37:37.791 INFO:teuthology.run_tasks:Running task console_log... 2023-06-08T03:37:37.895 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5056f08f70>, signals=[15]) 2023-06-08T03:37:37.896 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-08T03:37:37.902 INFO:teuthology.task.internal:Opening connections... 2023-06-08T03:37:37.902 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:37:37.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-08T03:37:37.971 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:37:37.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-08T03:37:38.082 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-08T03:37:38.088 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2023-06-08T03:37:38.109 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2023-06-08T03:37:38.110 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2023-06-08T03:37:38.169 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Red Hat Enterprise Linux" 2023-06-08T03:37:38.169 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="8.6 (Ootpa)" 2023-06-08T03:37:38.169 INFO:teuthology.orchestra.run.smithi079.stdout:ID="rhel" 2023-06-08T03:37:38.170 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="fedora" 2023-06-08T03:37:38.170 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="8.6" 2023-06-08T03:37:38.170 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el8" 2023-06-08T03:37:38.170 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-08T03:37:38.171 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2023-06-08T03:37:38.171 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-08T03:37:38.171 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.redhat.com/" 2023-06-08T03:37:38.171 INFO:teuthology.orchestra.run.smithi079.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-08T03:37:38.171 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-08T03:37:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:37:38.172 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-08T03:37:38.172 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-08T03:37:38.172 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-08T03:37:38.172 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-08T03:37:38.173 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2023-06-08T03:37:38.199 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-06-08T03:37:38.219 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-06-08T03:37:38.219 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-06-08T03:37:38.275 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-06-08T03:37:38.276 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-06-08T03:37:38.276 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-06-08T03:37:38.276 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-06-08T03:37:38.276 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-06-08T03:37:38.277 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-06-08T03:37:38.277 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-08T03:37:38.277 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-06-08T03:37:38.277 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-08T03:37:38.277 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-08T03:37:38.278 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-08T03:37:38.279 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-06-08T03:37:38.300 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-08T03:37:38.309 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-08T03:37:38.335 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-08T03:37:38.336 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2023-06-08T03:37:38.339 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-06-08T03:37:38.356 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-08T03:37:38.361 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-08T03:37:38.362 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2023-06-08T03:37:38.398 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-06-08T03:37:38.425 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-08T03:37:38.488 INFO:teuthology.run_tasks:Running task kernel... 2023-06-08T03:37:38.504 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-08T03:37:38.504 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-08T03:37:38.505 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-08T03:37:38.506 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-08T03:37:38.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T03:37:38.521 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-06-08T03:37:38.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T03:37:38.525 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2023-06-08T03:37:38.577 INFO:teuthology.orchestra.run.smithi156.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:38.577 INFO:teuthology.task.kernel:Running kernel on smithi156: 4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:38.578 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum install -y kernel 2023-06-08T03:37:38.583 INFO:teuthology.orchestra.run.smithi079.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:38.583 INFO:teuthology.task.kernel:Running kernel on smithi079: 4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:38.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2023-06-08T03:37:39.754 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-08T03:37:39.754 INFO:teuthology.orchestra.run.smithi079.stdout:Unable to read consumer identity 2023-06-08T03:37:40.203 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-06-08T03:37:40.204 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-06-08T03:37:43.356 INFO:teuthology.orchestra.run.smithi079.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-08T03:37:44.265 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-08T03:37:44.373 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-06-08T03:37:45.049 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-06-08T03:37:48.098 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 72 kB/s | 24 kB 00:00 2023-06-08T03:37:48.433 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 580 kB/s | 24 kB 00:00 2023-06-08T03:37:48.717 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-08T03:37:48.765 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-06-08T03:37:48.766 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2023-06-08T03:37:48.767 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-06-08T03:37:48.895 DEBUG:teuthology.orchestra.run.smithi079:> echo no | sudo yum reinstall kernel || true 2023-06-08T03:37:49.094 INFO:teuthology.orchestra.run.smithi156.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-08T03:37:49.141 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-06-08T03:37:49.142 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2023-06-08T03:37:49.142 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-06-08T03:37:49.252 DEBUG:teuthology.orchestra.run.smithi156:> echo no | sudo yum reinstall kernel || true 2023-06-08T03:37:49.263 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-08T03:37:49.263 INFO:teuthology.orchestra.run.smithi079.stdout:Unable to read consumer identity 2023-06-08T03:37:49.519 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:02 ago on Thu 08 Jun 2023 03:37:47 AM UTC. 2023-06-08T03:37:49.559 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-08T03:37:49.583 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-06-08T03:37:49.584 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-06-08T03:37:49.590 INFO:teuthology.orchestra.run.smithi079.stderr:Error: No packages marked for reinstall. 2023-06-08T03:37:49.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum reinstall -y kernel || true 2023-06-08T03:37:49.849 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:01 ago on Thu 08 Jun 2023 03:37:48 AM UTC. 2023-06-08T03:37:49.896 INFO:teuthology.orchestra.run.smithi156.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-08T03:37:49.936 INFO:teuthology.orchestra.run.smithi156.stderr:Error: No packages marked for reinstall. 2023-06-08T03:37:49.965 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-08T03:37:49.965 INFO:teuthology.orchestra.run.smithi079.stdout:Unable to read consumer identity 2023-06-08T03:37:49.983 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum reinstall -y kernel || true 2023-06-08T03:37:50.217 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:03 ago on Thu 08 Jun 2023 03:37:47 AM UTC. 2023-06-08T03:37:50.256 INFO:teuthology.orchestra.run.smithi079.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-08T03:37:50.284 INFO:teuthology.orchestra.run.smithi079.stderr:Error: No packages marked for reinstall. 2023-06-08T03:37:50.313 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-06-08T03:37:50.314 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-06-08T03:37:50.333 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2023-06-08T03:37:50.415 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:50.416 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:50.416 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-08T03:37:50.416 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-08T03:37:50.416 DEBUG:teuthology.orchestra.run.smithi079:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-08T03:37:50.453 INFO:teuthology.orchestra.run.smithi079.stdout:ttyS1 2023-06-08T03:37:50.474 DEBUG:teuthology.parallel:result is None 2023-06-08T03:37:50.567 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:00:02 ago on Thu 08 Jun 2023 03:37:48 AM UTC. 2023-06-08T03:37:50.610 INFO:teuthology.orchestra.run.smithi156.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-08T03:37:50.641 INFO:teuthology.orchestra.run.smithi156.stderr:Error: No packages marked for reinstall. 2023-06-08T03:37:50.689 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q kernel | sort -rV | head -n 1 2023-06-08T03:37:50.769 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:50.769 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-08T03:37:50.769 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-08T03:37:50.769 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-06-08T03:37:50.770 DEBUG:teuthology.orchestra.run.smithi156:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-08T03:37:50.802 INFO:teuthology.orchestra.run.smithi156.stdout:ttyS1 2023-06-08T03:37:50.820 DEBUG:teuthology.parallel:result is None 2023-06-08T03:37:50.821 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-08T03:37:50.831 INFO:teuthology.task.internal:Creating test directory... 2023-06-08T03:37:50.831 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-08T03:37:50.834 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-08T03:37:50.876 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-08T03:37:50.883 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-08T03:37:50.901 INFO:teuthology.task.internal:Creating archive directory... 2023-06-08T03:37:50.901 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-08T03:37:50.904 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-08T03:37:50.939 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-08T03:37:50.955 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-08T03:37:50.956 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-08T03:37:50.962 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-08T03:37:50.991 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-08T03:37:51.002 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-08T03:37:51.011 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-08T03:37:51.021 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-08T03:37:51.022 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-08T03:37:51.032 INFO:teuthology.task.internal:Configuring sudo... 2023-06-08T03:37:51.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-08T03:37:51.048 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-08T03:37:51.089 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-08T03:37:51.102 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-08T03:37:51.102 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-08T03:37:51.114 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-08T03:37:51.145 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-08T03:37:51.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-08T03:37:51.261 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-08T03:37:51.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-08T03:37:51.385 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:37:51.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-08T03:37:51.453 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-08T03:37:51.498 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-08T03:37:51.546 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-08T03:37:51.633 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-08T03:37:51.672 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:37:51.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-08T03:37:51.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2023-06-08T03:37:51.741 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-06-08T03:37:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-08T03:37:51.844 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-08T03:37:52.313 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-08T03:37:52.328 INFO:teuthology.task.internal:Starting timer... 2023-06-08T03:37:52.328 INFO:teuthology.run_tasks:Running task pcp... 2023-06-08T03:37:52.432 INFO:teuthology.run_tasks:Running task selinux... 2023-06-08T03:37:52.454 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-08T03:37:52.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2023-06-08T03:37:52.517 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2023-06-08T03:37:52.519 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-06-08T03:37:52.573 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-06-08T03:37:52.575 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-08T03:37:52.575 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2023-06-08T03:37:52.603 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2023-06-08T03:37:52.603 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-06-08T03:37:52.622 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-06-08T03:37:52.623 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive'} 2023-06-08T03:37:52.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-08T03:37:52.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T03:37:52.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-08T03:37:52.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T03:37:52.701 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-08T03:37:52.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2023-06-08T03:37:52.740 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-06-08T03:37:52.776 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-08T03:37:52.784 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-08T03:37:52.785 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-08T03:37:52.807 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-08T03:37:52.808 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-08T03:46:29.708 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-06-08T03:46:29.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2023-06-08T03:46:29.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-08T03:46:29.780 DEBUG:teuthology.orchestra.run.smithi079:> true 2023-06-08T03:46:29.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-06-08T03:46:29.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-08T03:46:29.915 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-06-08T03:46:29.986 INFO:teuthology.run_tasks:Running task clock... 2023-06-08T03:46:30.006 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-08T03:46:30.006 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-08T03:46:30.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-08T03:46:30.009 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-08T03:46:30.010 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-08T03:46:30.041 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-08T03:46:30.056 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-08T03:46:30.059 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-08T03:46:30.074 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-08T03:46:30.124 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2023-06-08T03:46:30.136 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2023-06-08T03:46:30.147 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-06-08T03:46:30.150 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-08T03:46:30.158 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-06-08T03:46:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-08T03:46:30.172 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-08T03:46:30.187 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-08T03:46:30.244 INFO:teuthology.orchestra.run.smithi079.stderr:bash: ntpq: command not found 2023-06-08T03:46:30.247 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-08T03:46:30.247 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2023-06-08T03:46:30.247 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.248 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.248 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.248 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.258 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-06-08T03:46:30.263 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-08T03:46:30.263 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-06-08T03:46:30.263 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.263 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.263 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.264 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-08T03:46:30.264 INFO:teuthology.run_tasks:Running task install... 2023-06-08T03:46:30.272 DEBUG:teuthology.task.install:project ceph 2023-06-08T03:46:30.273 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}} 2023-06-08T03:46:30.273 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'} 2023-06-08T03:46:30.273 INFO:teuthology.task.install:Using flavor: default 2023-06-08T03:46:30.279 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-08T03:46:30.280 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-08T03:46:30.281 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'tag': None, 'wait_for_package': False} 2023-06-08T03:46:30.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-08T03:46:30.283 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'tag': None, 'wait_for_package': False} 2023-06-08T03:46:30.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-08T03:46:30.430 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/ 2023-06-08T03:46:30.430 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4313.ga02638f8 2023-06-08T03:46:30.434 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/ 2023-06-08T03:46:30.435 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4313.ga02638f8 2023-06-08T03:46:30.733 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-08T03:46:30.734 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:46:30.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-08T03:46:30.761 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-08T03:46:30.762 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a02638f808b8fe11bccdd01222c0e6e2eb71120b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-08T03:46:30.814 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lxb-fscrypt-20230607-0901/a02638f808b8fe11bccdd01222c0e6e2eb71120b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-08T03:46:30.814 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:46:30.815 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-08T03:46:30.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-08T03:46:30.846 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-08T03:46:30.847 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a02638f808b8fe11bccdd01222c0e6e2eb71120b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-08T03:46:30.908 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-08T03:46:30.915 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-08T03:46:30.974 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2023-06-08T03:46:30.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2023-06-08T03:46:30.994 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-08T03:46:31.060 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-06-08T03:46:31.061 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-06-08T03:46:31.344 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-08T03:46:31.435 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-06-08T03:46:31.608 INFO:teuthology.orchestra.run.smithi079.stdout:44 files removed 2023-06-08T03:46:31.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-08T03:46:31.675 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-06-08T03:46:31.721 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-08T03:46:31.982 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-08T03:46:32.052 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-06-08T03:46:32.764 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 186 kB/s | 80 kB 00:00 2023-06-08T03:46:32.833 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 193 kB/s | 80 kB 00:00 2023-06-08T03:46:33.431 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 15 kB/s | 9.4 kB 00:00 2023-06-08T03:46:33.592 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 13 kB/s | 9.4 kB 00:00 2023-06-08T03:46:34.132 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 2.2 kB/s | 1.5 kB 00:00 2023-06-08T03:46:34.226 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 2.4 kB/s | 1.5 kB 00:00 2023-06-08T03:46:36.640 INFO:teuthology.orchestra.run.smithi079.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-08T03:46:36.711 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-08T03:46:37.791 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-06-08T03:46:41.391 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 189 kB/s | 24 kB 00:00 2023-06-08T03:46:42.284 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-06-08T03:46:50.351 INFO:teuthology.orchestra.run.smithi079.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-06-08T03:46:52.042 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2023-06-08T03:46:55.518 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-06-08T03:46:56.461 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-06-08T03:47:01.289 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-08T03:47:01.290 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-08T03:47:01.304 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-08T03:47:01.305 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-08T03:47:01.307 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-08T03:47:01.308 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-08T03:47:01.310 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-08T03:47:01.313 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-08T03:47:01.314 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-08T03:47:01.447 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-06-08T03:47:01.459 INFO:teuthology.orchestra.run.smithi079.stdout:=========================================================================================================================== 2023-06-08T03:47:01.459 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-06-08T03:47:01.459 INFO:teuthology.orchestra.run.smithi079.stdout:=========================================================================================================================== 2023-06-08T03:47:01.459 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-06-08T03:47:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-08T03:47:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-08T03:47:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.4 k 2023-06-08T03:47:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 5.4 M 2023-06-08T03:47:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 848 k 2023-06-08T03:47:01.460 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 166 k 2023-06-08T03:47:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 1.5 M 2023-06-08T03:47:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 128 k 2023-06-08T03:47:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 1.7 M 2023-06-08T03:47:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 7.4 M 2023-06-08T03:47:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 47 k 2023-06-08T03:47:01.461 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 13 M 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 41 M 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 258 k 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 160 k 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 258 k 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 18 k 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-08T03:47:01.462 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 31 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 771 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 139 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-08T03:47:01.463 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 197 k 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 370 k 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 365 k 2023-06-08T03:47:01.464 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 107 k 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 94 k 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.4 M 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 179 k 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-08T03:47:01.465 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.7 M 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.6 M 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 22 M 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 23 k 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 2.3 M 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 240 k 2023-06-08T03:47:01.466 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 4.6 M 2023-06-08T03:47:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 18 M 2023-06-08T03:47:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 15 k 2023-06-08T03:47:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 24 k 2023-06-08T03:47:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-08T03:47:01.467 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 137 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-08T03:47:01.468 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-08T03:47:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-08T03:47:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-08T03:47:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-08T03:47:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 473 k 2023-06-08T03:47:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-08T03:47:01.469 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.5 M 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-08T03:47:01.470 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-08T03:47:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-08T03:47:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-08T03:47:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-08T03:47:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-08T03:47:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-08T03:47:01.471 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-08T03:47:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-08T03:47:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-08T03:47:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-08T03:47:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 45 k 2023-06-08T03:47:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 119 k 2023-06-08T03:47:01.472 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-08T03:47:01.473 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-08T03:47:01.474 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-08T03:47:01.475 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-08T03:47:01.476 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-08T03:47:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-08T03:47:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-08T03:47:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-08T03:47:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-08T03:47:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-08T03:47:01.477 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-08T03:47:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-08T03:47:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-08T03:47:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-08T03:47:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-08T03:47:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-08T03:47:01.478 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-08T03:47:01.479 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling module streams: 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql 10 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout:=========================================================================================================================== 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout:Install 128 Packages 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 3 Packages 2023-06-08T03:47:01.480 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:47:01.481 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 197 M 2023-06-08T03:47:01.481 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-06-08T03:47:01.917 INFO:teuthology.orchestra.run.smithi079.stdout:(1/131): ceph-18.0.0-4313.ga02638f8.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-06-08T03:47:03.034 INFO:teuthology.orchestra.run.smithi079.stdout:(2/131): ceph-fuse-18.0.0-4313.ga02638f8.el8.x8 760 kB/s | 848 kB 00:01 2023-06-08T03:47:03.252 INFO:teuthology.orchestra.run.smithi079.stdout:(3/131): ceph-immutable-object-cache-18.0.0-431 765 kB/s | 166 kB 00:00 2023-06-08T03:47:04.518 INFO:teuthology.orchestra.run.smithi079.stdout:(4/131): ceph-base-18.0.0-4313.ga02638f8.el8.x8 1.9 MB/s | 5.4 MB 00:02 2023-06-08T03:47:04.553 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-06-08T03:47:05.285 INFO:teuthology.orchestra.run.smithi079.stdout:(5/131): ceph-mgr-18.0.0-4313.ga02638f8.el8.x86 2.0 MB/s | 1.5 MB 00:00 2023-06-08T03:47:05.610 INFO:teuthology.orchestra.run.smithi079.stdout:(6/131): ceph-mds-18.0.0-4313.ga02638f8.el8.x86 992 kB/s | 2.3 MB 00:02 2023-06-08T03:47:07.526 INFO:teuthology.orchestra.run.smithi079.stdout:(7/131): ceph-mon-18.0.0-4313.ga02638f8.el8.x86 2.1 MB/s | 4.6 MB 00:02 2023-06-08T03:47:13.376 INFO:teuthology.orchestra.run.smithi079.stdout:(8/131): ceph-radosgw-18.0.0-4313.ga02638f8.el8 2.2 MB/s | 13 MB 00:05 2023-06-08T03:47:13.501 INFO:teuthology.orchestra.run.smithi079.stdout:(9/131): ceph-selinux-18.0.0-4313.ga02638f8.el8 195 kB/s | 24 kB 00:00 2023-06-08T03:47:14.501 INFO:teuthology.orchestra.run.smithi079.stdout:(10/131): ceph-osd-18.0.0-4313.ga02638f8.el8.x8 2.0 MB/s | 18 MB 00:08 2023-06-08T03:47:14.660 INFO:teuthology.orchestra.run.smithi079.stdout:(11/131): cephfs-mirror-18.0.0-4313.ga02638f8.e 1.6 MB/s | 258 kB 00:00 2023-06-08T03:47:14.752 INFO:teuthology.orchestra.run.smithi079.stdout:(12/131): libcephfs-devel-18.0.0-4313.ga02638f8 338 kB/s | 31 kB 00:00 2023-06-08T03:47:15.110 INFO:teuthology.orchestra.run.smithi079.stdout:(13/131): libcephfs2-18.0.0-4313.ga02638f8.el8. 2.1 MB/s | 771 kB 00:00 2023-06-08T03:47:15.202 INFO:teuthology.orchestra.run.smithi079.stdout:(14/131): libcephsqlite-18.0.0-4313.ga02638f8.e 1.5 MB/s | 137 kB 00:00 2023-06-08T03:47:15.310 INFO:teuthology.orchestra.run.smithi079.stdout:(15/131): librados-devel-18.0.0-4313.ga02638f8. 1.3 MB/s | 139 kB 00:00 2023-06-08T03:47:15.484 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-08T03:47:15.485 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-08T03:47:15.499 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-08T03:47:15.500 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-08T03:47:15.502 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-08T03:47:15.503 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-08T03:47:15.505 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-08T03:47:15.508 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-08T03:47:15.509 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-08T03:47:15.528 INFO:teuthology.orchestra.run.smithi079.stdout:(16/131): libradosstriper1-18.0.0-4313.ga02638f 2.1 MB/s | 473 kB 00:00 2023-06-08T03:47:15.641 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-06-08T03:47:15.654 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.4 k 2023-06-08T03:47:15.655 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 5.4 M 2023-06-08T03:47:15.656 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 848 k 2023-06-08T03:47:15.656 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 166 k 2023-06-08T03:47:15.656 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 1.5 M 2023-06-08T03:47:15.656 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 128 k 2023-06-08T03:47:15.656 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 1.7 M 2023-06-08T03:47:15.656 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 7.4 M 2023-06-08T03:47:15.657 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 47 k 2023-06-08T03:47:15.657 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 13 M 2023-06-08T03:47:15.657 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 41 M 2023-06-08T03:47:15.657 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 258 k 2023-06-08T03:47:15.657 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 160 k 2023-06-08T03:47:15.658 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 258 k 2023-06-08T03:47:15.658 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 18 k 2023-06-08T03:47:15.658 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-08T03:47:15.658 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-08T03:47:15.658 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-08T03:47:15.658 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 31 k 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 771 k 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 139 k 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-08T03:47:15.659 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-08T03:47:15.660 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-08T03:47:15.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 197 k 2023-06-08T03:47:15.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 370 k 2023-06-08T03:47:15.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 365 k 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 107 k 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 94 k 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.4 M 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 179 k 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-06-08T03:47:15.661 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.7 M 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 3.6 M 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 22 M 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 23 k 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 2.3 M 2023-06-08T03:47:15.662 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 240 k 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 4.6 M 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 18 M 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4313.ga02638f8.el8 ceph-noarch 15 k 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 24 k 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-08T03:47:15.663 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-08T03:47:15.664 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-08T03:47:15.664 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-08T03:47:15.664 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 137 k 2023-06-08T03:47:15.665 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-08T03:47:15.665 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-08T03:47:15.665 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-08T03:47:15.665 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-08T03:47:15.666 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-08T03:47:15.666 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-08T03:47:15.666 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 473 k 2023-06-08T03:47:15.667 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-08T03:47:15.667 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 6.5 M 2023-06-08T03:47:15.667 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-08T03:47:15.667 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-08T03:47:15.668 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-08T03:47:15.668 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-08T03:47:15.668 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-08T03:47:15.669 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-08T03:47:15.669 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-08T03:47:15.669 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-08T03:47:15.670 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-08T03:47:15.670 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-08T03:47:15.670 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-08T03:47:15.670 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-08T03:47:15.671 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-08T03:47:15.671 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-08T03:47:15.671 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-08T03:47:15.672 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-08T03:47:15.672 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 45 k 2023-06-08T03:47:15.673 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-4313.ga02638f8.el8 ceph 119 k 2023-06-08T03:47:15.673 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-08T03:47:15.673 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-08T03:47:15.673 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-08T03:47:15.674 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-08T03:47:15.674 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-08T03:47:15.674 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-08T03:47:15.674 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-08T03:47:15.675 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-08T03:47:15.675 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-08T03:47:15.675 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-08T03:47:15.676 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-08T03:47:15.676 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-08T03:47:15.676 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-08T03:47:15.676 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-08T03:47:15.677 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-08T03:47:15.677 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-08T03:47:15.677 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-08T03:47:15.678 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-08T03:47:15.678 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-08T03:47:15.678 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-08T03:47:15.678 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-08T03:47:15.678 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-08T03:47:15.678 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-08T03:47:15.679 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-08T03:47:15.679 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-08T03:47:15.679 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-08T03:47:15.679 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-08T03:47:15.679 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-08T03:47:15.680 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-08T03:47:15.680 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-08T03:47:15.680 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-08T03:47:15.680 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-08T03:47:15.680 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-08T03:47:15.681 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-08T03:47:15.681 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-08T03:47:15.681 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-08T03:47:15.681 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-08T03:47:15.681 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-08T03:47:15.681 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-08T03:47:15.682 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-08T03:47:15.682 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-08T03:47:15.682 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-08T03:47:15.682 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-08T03:47:15.683 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-08T03:47:15.683 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-08T03:47:15.683 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-08T03:47:15.683 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-08T03:47:15.684 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-08T03:47:15.684 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-08T03:47:15.684 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-06-08T03:47:15.684 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-06-08T03:47:15.685 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:47:15.685 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-06-08T03:47:15.685 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-06-08T03:47:15.686 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-06-08T03:47:15.686 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-06-08T03:47:15.686 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:47:15.687 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 197 M 2023-06-08T03:47:15.687 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-06-08T03:47:16.145 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-4313.ga02638f8.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-06-08T03:47:16.871 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-4313.ga02638f8.el8.x8 1.1 MB/s | 848 kB 00:00 2023-06-08T03:47:17.071 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-431 829 kB/s | 166 kB 00:00 2023-06-08T03:47:18.129 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-base-18.0.0-4313.ga02638f8.el8.x8 2.3 MB/s | 5.4 MB 00:02 2023-06-08T03:47:18.279 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-mds-18.0.0-4313.ga02638f8.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-06-08T03:47:18.838 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-mgr-18.0.0-4313.ga02638f8.el8.x86 2.2 MB/s | 1.5 MB 00:00 2023-06-08T03:47:19.094 INFO:teuthology.orchestra.run.smithi079.stdout:(17/131): ceph-common-18.0.0-4313.ga02638f8.el8 1.2 MB/s | 22 MB 00:17 2023-06-08T03:47:19.369 INFO:teuthology.orchestra.run.smithi079.stdout:(18/131): python3-ceph-argparse-18.0.0-4313.ga0 163 kB/s | 45 kB 00:00 2023-06-08T03:47:19.661 INFO:teuthology.orchestra.run.smithi079.stdout:(19/131): python3-ceph-common-18.0.0-4313.ga026 407 kB/s | 119 kB 00:00 2023-06-08T03:47:20.169 INFO:teuthology.orchestra.run.smithi079.stdout:(20/131): python3-cephfs-18.0.0-4313.ga02638f8. 388 kB/s | 197 kB 00:00 2023-06-08T03:47:20.336 INFO:teuthology.orchestra.run.smithi079.stdout:(21/131): librgw2-18.0.0-4313.ga02638f8.el8.x86 1.4 MB/s | 6.5 MB 00:04 2023-06-08T03:47:20.711 INFO:teuthology.orchestra.run.smithi079.stdout:(22/131): python3-rbd-18.0.0-4313.ga02638f8.el8 974 kB/s | 365 kB 00:00 2023-06-08T03:47:20.870 INFO:teuthology.orchestra.run.smithi079.stdout:(23/131): python3-rgw-18.0.0-4313.ga02638f8.el8 674 kB/s | 107 kB 00:00 2023-06-08T03:47:20.945 INFO:teuthology.orchestra.run.smithi079.stdout:(24/131): python3-rados-18.0.0-4313.ga02638f8.e 477 kB/s | 370 kB 00:00 2023-06-08T03:47:21.037 INFO:teuthology.orchestra.run.smithi079.stdout:(25/131): rbd-fuse-18.0.0-4313.ga02638f8.el8.x8 566 kB/s | 94 kB 00:00 2023-06-08T03:47:21.279 INFO:teuthology.orchestra.run.smithi079.stdout:(26/131): rbd-nbd-18.0.0-4313.ga02638f8.el8.x86 742 kB/s | 179 kB 00:00 2023-06-08T03:47:21.330 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-4313.ga02638f8.el8.x86 1.5 MB/s | 4.6 MB 00:03 2023-06-08T03:47:21.380 INFO:teuthology.orchestra.run.smithi079.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4313.g 230 kB/s | 23 kB 00:00 2023-06-08T03:47:21.555 INFO:teuthology.orchestra.run.smithi079.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4313.ga02638f 731 kB/s | 128 kB 00:00 2023-06-08T03:47:23.072 INFO:teuthology.orchestra.run.smithi079.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4313.ga0263 1.1 MB/s | 1.7 MB 00:01 2023-06-08T03:47:26.213 INFO:teuthology.orchestra.run.smithi079.stdout:(30/131): rbd-mirror-18.0.0-4313.ga02638f8.el8. 670 kB/s | 3.4 MB 00:05 2023-06-08T03:47:26.738 INFO:teuthology.orchestra.run.smithi079.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4313.ga0 458 kB/s | 240 kB 00:00 2023-06-08T03:47:26.947 INFO:teuthology.orchestra.run.smithi079.stdout:(32/131): ceph-mgr-rook-18.0.0-4313.ga02638f8.e 228 kB/s | 47 kB 00:00 2023-06-08T03:47:26.955 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-osd-18.0.0-4313.ga02638f8.el8.x86 2.2 MB/s | 18 MB 00:08 2023-06-08T03:47:27.039 INFO:teuthology.orchestra.run.smithi079.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4313.ga 159 kB/s | 15 kB 00:00 2023-06-08T03:47:27.047 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-4313.ga02638f8.el8 266 kB/s | 24 kB 00:00 2023-06-08T03:47:27.706 INFO:teuthology.orchestra.run.smithi079.stdout:(34/131): ceph-volume-18.0.0-4313.ga02638f8.el8 387 kB/s | 258 kB 00:00 2023-06-08T03:47:28.106 INFO:teuthology.orchestra.run.smithi079.stdout:(35/131): cephadm-18.0.0-4313.ga02638f8.el8.noa 400 kB/s | 160 kB 00:00 2023-06-08T03:47:28.206 INFO:teuthology.orchestra.run.smithi079.stdout:(36/131): cephfs-top-18.0.0-4313.ga02638f8.el8. 180 kB/s | 18 kB 00:00 2023-06-08T03:47:28.706 INFO:teuthology.orchestra.run.smithi079.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 1.3 MB/s | 7.4 MB 00:05 2023-06-08T03:47:28.755 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-radosgw-18.0.0-4313.ga02638f8.el 1.8 MB/s | 13 MB 00:07 2023-06-08T03:47:28.857 INFO:teuthology.orchestra.run.smithi079.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 471 kB/s | 306 kB 00:00 2023-06-08T03:47:28.914 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-4313.ga02638f8.e 1.6 MB/s | 258 kB 00:00 2023-06-08T03:47:28.982 INFO:teuthology.orchestra.run.smithi079.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 601 kB/s | 75 kB 00:00 2023-06-08T03:47:28.998 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-4313.ga02638f8 372 kB/s | 31 kB 00:00 2023-06-08T03:47:29.124 INFO:teuthology.orchestra.run.smithi079.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 141 kB/s | 59 kB 00:00 2023-06-08T03:47:29.191 INFO:teuthology.orchestra.run.smithi079.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-06-08T03:47:29.249 INFO:teuthology.orchestra.run.smithi079.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 348 kB/s | 44 kB 00:00 2023-06-08T03:47:29.308 INFO:teuthology.orchestra.run.smithi079.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2023-06-08T03:47:29.375 INFO:teuthology.orchestra.run.smithi079.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2023-06-08T03:47:29.406 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-4313.ga02638f8.el8. 1.8 MB/s | 771 kB 00:00 2023-06-08T03:47:29.408 INFO:teuthology.orchestra.run.smithi079.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-06-08T03:47:29.506 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-4313.ga02638f8.e 1.3 MB/s | 137 kB 00:00 2023-06-08T03:47:29.542 INFO:teuthology.orchestra.run.smithi079.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2023-06-08T03:47:29.593 INFO:teuthology.orchestra.run.smithi079.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 794 kB/s | 173 kB 00:00 2023-06-08T03:47:29.638 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-4313.ga02638f8. 1.0 MB/s | 139 kB 00:00 2023-06-08T03:47:29.651 INFO:teuthology.orchestra.run.smithi079.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 749 kB/s | 81 kB 00:00 2023-06-08T03:47:29.693 INFO:teuthology.orchestra.run.smithi079.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-06-08T03:47:29.752 INFO:teuthology.orchestra.run.smithi079.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-06-08T03:47:29.860 INFO:teuthology.orchestra.run.smithi079.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-06-08T03:47:29.957 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-4313.ga02638f 1.4 MB/s | 473 kB 00:00 2023-06-08T03:47:29.977 INFO:teuthology.orchestra.run.smithi079.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 505 kB/s | 59 kB 00:00 2023-06-08T03:47:30.020 INFO:teuthology.orchestra.run.smithi079.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.6 MB/s | 1.2 MB 00:00 2023-06-08T03:47:30.078 INFO:teuthology.orchestra.run.smithi079.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 599 kB/s | 60 kB 00:00 2023-06-08T03:47:30.137 INFO:teuthology.orchestra.run.smithi079.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-06-08T03:47:30.179 INFO:teuthology.orchestra.run.smithi079.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-06-08T03:47:30.237 INFO:teuthology.orchestra.run.smithi079.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-06-08T03:47:30.296 INFO:teuthology.orchestra.run.smithi079.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-06-08T03:47:30.338 INFO:teuthology.orchestra.run.smithi079.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 614 kB/s | 61 kB 00:00 2023-06-08T03:47:30.397 INFO:teuthology.orchestra.run.smithi079.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-06-08T03:47:30.438 INFO:teuthology.orchestra.run.smithi079.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2023-06-08T03:47:30.522 INFO:teuthology.orchestra.run.smithi079.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 234 kB/s | 29 kB 00:00 2023-06-08T03:47:30.547 INFO:teuthology.orchestra.run.smithi079.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-06-08T03:47:30.648 INFO:teuthology.orchestra.run.smithi079.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-06-08T03:47:30.673 INFO:teuthology.orchestra.run.smithi079.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-06-08T03:47:30.757 INFO:teuthology.orchestra.run.smithi079.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-06-08T03:47:30.782 INFO:teuthology.orchestra.run.smithi079.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 787 kB/s | 85 kB 00:00 2023-06-08T03:47:30.857 INFO:teuthology.orchestra.run.smithi079.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-06-08T03:47:30.882 INFO:teuthology.orchestra.run.smithi079.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-06-08T03:47:30.941 INFO:teuthology.orchestra.run.smithi079.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 802 kB/s | 47 kB 00:00 2023-06-08T03:47:30.983 INFO:teuthology.orchestra.run.smithi079.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2023-06-08T03:47:31.008 INFO:teuthology.orchestra.run.smithi079.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-06-08T03:47:31.050 INFO:teuthology.orchestra.run.smithi079.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 637 kB/s | 43 kB 00:00 2023-06-08T03:47:31.075 INFO:teuthology.orchestra.run.smithi079.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.5 MB/s | 220 kB 00:00 2023-06-08T03:47:31.092 INFO:teuthology.orchestra.run.smithi079.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-06-08T03:47:31.109 INFO:teuthology.orchestra.run.smithi079.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-06-08T03:47:31.126 INFO:teuthology.orchestra.run.smithi079.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.2 MB/s | 43 kB 00:00 2023-06-08T03:47:31.151 INFO:teuthology.orchestra.run.smithi079.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.5 MB/s | 109 kB 00:00 2023-06-08T03:47:31.177 INFO:teuthology.orchestra.run.smithi079.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-06-08T03:47:31.210 INFO:teuthology.orchestra.run.smithi079.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-06-08T03:47:31.227 INFO:teuthology.orchestra.run.smithi079.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-06-08T03:47:31.252 INFO:teuthology.orchestra.run.smithi079.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-06-08T03:47:31.286 INFO:teuthology.orchestra.run.smithi079.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-06-08T03:47:31.312 INFO:teuthology.orchestra.run.smithi079.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 304 kB/s | 18 kB 00:00 2023-06-08T03:47:31.345 INFO:teuthology.orchestra.run.smithi079.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-06-08T03:47:31.370 INFO:teuthology.orchestra.run.smithi079.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-08T03:47:31.413 INFO:teuthology.orchestra.run.smithi079.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-06-08T03:47:31.471 INFO:teuthology.orchestra.run.smithi079.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2023-06-08T03:47:31.496 INFO:teuthology.orchestra.run.smithi079.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-06-08T03:47:31.522 INFO:teuthology.orchestra.run.smithi079.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-06-08T03:47:31.555 INFO:teuthology.orchestra.run.smithi079.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.6 MB/s | 98 kB 00:00 2023-06-08T03:47:31.581 INFO:teuthology.orchestra.run.smithi079.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 985 kB/s | 58 kB 00:00 2023-06-08T03:47:31.606 INFO:teuthology.orchestra.run.smithi079.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-06-08T03:47:31.639 INFO:teuthology.orchestra.run.smithi079.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-06-08T03:47:31.665 INFO:teuthology.orchestra.run.smithi079.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-06-08T03:47:31.690 INFO:teuthology.orchestra.run.smithi079.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-08T03:47:31.749 INFO:teuthology.orchestra.run.smithi079.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-06-08T03:47:31.782 INFO:teuthology.orchestra.run.smithi079.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-06-08T03:47:31.824 INFO:teuthology.orchestra.run.smithi079.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-06-08T03:47:31.866 INFO:teuthology.orchestra.run.smithi079.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-06-08T03:47:31.900 INFO:teuthology.orchestra.run.smithi079.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-06-08T03:47:31.950 INFO:teuthology.orchestra.run.smithi079.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-06-08T03:47:31.992 INFO:teuthology.orchestra.run.smithi079.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-06-08T03:47:32.034 INFO:teuthology.orchestra.run.smithi079.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-06-08T03:47:32.076 INFO:teuthology.orchestra.run.smithi079.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-06-08T03:47:32.226 INFO:teuthology.orchestra.run.smithi079.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2023-06-08T03:47:32.268 INFO:teuthology.orchestra.run.smithi079.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.6 MB/s | 713 kB 00:00 2023-06-08T03:47:32.302 INFO:teuthology.orchestra.run.smithi079.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 801 kB/s | 27 kB 00:00 2023-06-08T03:47:32.343 INFO:teuthology.orchestra.run.smithi079.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-08T03:47:32.369 INFO:teuthology.orchestra.run.smithi079.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-06-08T03:47:32.444 INFO:teuthology.orchestra.run.smithi079.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-06-08T03:47:32.486 INFO:teuthology.orchestra.run.smithi079.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-06-08T03:47:32.553 INFO:teuthology.orchestra.run.smithi079.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2023-06-08T03:47:32.578 INFO:teuthology.orchestra.run.smithi079.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-06-08T03:47:32.604 INFO:teuthology.orchestra.run.smithi079.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-06-08T03:47:32.629 INFO:teuthology.orchestra.run.smithi079.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-06-08T03:47:32.654 INFO:teuthology.orchestra.run.smithi079.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 396 kB/s | 20 kB 00:00 2023-06-08T03:47:32.671 INFO:teuthology.orchestra.run.smithi079.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-06-08T03:47:32.705 INFO:teuthology.orchestra.run.smithi079.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-06-08T03:47:32.747 INFO:teuthology.orchestra.run.smithi079.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-06-08T03:47:32.839 INFO:teuthology.orchestra.run.smithi079.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 29 MB/s | 4.9 MB 00:00 2023-06-08T03:47:32.881 INFO:teuthology.orchestra.run.smithi079.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-06-08T03:47:32.981 INFO:teuthology.orchestra.run.smithi079.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-06-08T03:47:33.006 INFO:teuthology.orchestra.run.smithi079.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-06-08T03:47:33.065 INFO:teuthology.orchestra.run.smithi079.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-06-08T03:47:33.157 INFO:teuthology.orchestra.run.smithi079.stdout:(126/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-06-08T03:47:33.199 INFO:teuthology.orchestra.run.smithi079.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-06-08T03:47:35.615 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): librgw2-18.0.0-4313.ga02638f8.el8.x86 1.2 MB/s | 6.5 MB 00:05 2023-06-08T03:47:35.624 INFO:teuthology.orchestra.run.smithi079.stdout:(128/131): librados2-18.0.0-4313.ga02638f8.el8. 1.5 MB/s | 3.7 MB 00:02 2023-06-08T03:47:35.674 INFO:teuthology.orchestra.run.smithi079.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-06-08T03:47:35.723 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-4313.ga0 413 kB/s | 45 kB 00:00 2023-06-08T03:47:35.890 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-4313.ga026 712 kB/s | 119 kB 00:00 2023-06-08T03:47:36.124 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-4313.ga02638f8. 846 kB/s | 197 kB 00:00 2023-06-08T03:47:36.441 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): python3-rados-18.0.0-4313.ga02638f8.e 1.1 MB/s | 370 kB 00:00 2023-06-08T03:47:36.758 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rbd-18.0.0-4313.ga02638f8.el8 1.1 MB/s | 365 kB 00:00 2023-06-08T03:47:36.933 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-4313.ga02638f8.el8 610 kB/s | 107 kB 00:00 2023-06-08T03:47:37.034 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): rbd-fuse-18.0.0-4313.ga02638f8.el8.x8 947 kB/s | 94 kB 00:00 2023-06-08T03:47:39.291 INFO:teuthology.orchestra.run.smithi079.stdout:(130/131): librbd1-18.0.0-4313.ga02638f8.el8.x8 598 kB/s | 3.6 MB 00:06 2023-06-08T03:47:39.509 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): rbd-mirror-18.0.0-4313.ga02638f8.el8. 1.4 MB/s | 3.4 MB 00:02 2023-06-08T03:47:39.684 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-nbd-18.0.0-4313.ga02638f8.el8.x86 1.0 MB/s | 179 kB 00:00 2023-06-08T03:47:39.784 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4313.g 231 kB/s | 23 kB 00:00 2023-06-08T03:47:39.960 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4313.ga02638f 730 kB/s | 128 kB 00:00 2023-06-08T03:47:41.268 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4313.ga0263 1.3 MB/s | 1.7 MB 00:01 2023-06-08T03:47:46.877 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 1.3 MB/s | 7.4 MB 00:05 2023-06-08T03:47:47.060 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4313.ga0 1.3 MB/s | 240 kB 00:00 2023-06-08T03:47:47.152 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-4313.ga02638f8.e 517 kB/s | 47 kB 00:00 2023-06-08T03:47:47.235 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4313.ga 174 kB/s | 15 kB 00:00 2023-06-08T03:47:47.478 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-4313.ga02638f8.el8 1.0 MB/s | 258 kB 00:00 2023-06-08T03:47:47.653 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-4313.ga02638f8.el8.noa 914 kB/s | 160 kB 00:00 2023-06-08T03:47:47.745 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephfs-top-18.0.0-4313.ga02638f8.el8. 196 kB/s | 18 kB 00:00 2023-06-08T03:47:48.220 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 645 kB/s | 306 kB 00:00 2023-06-08T03:47:48.304 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 706 kB/s | 59 kB 00:00 2023-06-08T03:47:48.387 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 902 kB/s | 75 kB 00:00 2023-06-08T03:47:48.563 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.7 MB/s | 476 kB 00:00 2023-06-08T03:47:48.646 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 524 kB/s | 44 kB 00:00 2023-06-08T03:47:48.774 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-06-08T03:47:48.880 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 311 kB/s | 33 kB 00:00 2023-06-08T03:47:48.972 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2023-06-08T03:47:49.089 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-06-08T03:47:49.190 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2023-06-08T03:47:49.290 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 813 kB/s | 81 kB 00:00 2023-06-08T03:47:49.390 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-06-08T03:47:49.490 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-06-08T03:47:49.682 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2023-06-08T03:47:49.783 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2023-06-08T03:47:49.883 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 587 kB/s | 59 kB 00:00 2023-06-08T03:47:49.992 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 556 kB/s | 60 kB 00:00 2023-06-08T03:47:50.092 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-06-08T03:47:50.309 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 72 kB/s | 16 kB 00:00 2023-06-08T03:47:50.401 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 367 kB/s | 34 kB 00:00 2023-06-08T03:47:50.502 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-06-08T03:47:50.602 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 615 kB/s | 61 kB 00:00 2023-06-08T03:47:50.694 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-06-08T03:47:50.786 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 261 kB/s | 24 kB 00:00 2023-06-08T03:47:50.886 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 293 kB/s | 29 kB 00:00 2023-06-08T03:47:50.978 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 300 kB/s | 27 kB 00:00 2023-06-08T03:47:51.079 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-06-08T03:47:51.187 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-06-08T03:47:51.279 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 668 kB/s | 61 kB 00:00 2023-06-08T03:47:51.371 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 930 kB/s | 85 kB 00:00 2023-06-08T03:47:51.463 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): ceph-common-18.0.0-4313.ga02638f8.el8 622 kB/s | 22 MB 00:35 2023-06-08T03:47:51.480 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2023-06-08T03:47:51.497 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-06-08T03:47:51.556 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 802 kB/s | 47 kB 00:00 2023-06-08T03:47:51.598 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-06-08T03:47:51.631 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-06-08T03:47:51.673 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-06-08T03:47:51.723 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2023-06-08T03:47:51.765 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-06-08T03:47:51.799 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.3 MB/s | 43 kB 00:00 2023-06-08T03:47:51.841 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-06-08T03:47:51.883 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-06-08T03:47:51.933 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-06-08T03:47:52.007 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 256 kB/s | 19 kB 00:00 2023-06-08T03:47:52.051 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.5 MB/s | 155 kB 00:00 2023-06-08T03:47:52.110 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-06-08T03:47:52.143 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 540 kB/s | 18 kB 00:00 2023-06-08T03:47:52.177 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2023-06-08T03:47:52.210 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-06-08T03:47:52.261 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-06-08T03:47:52.319 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-06-08T03:47:52.369 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-06-08T03:47:52.395 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-06-08T03:47:52.421 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-06-08T03:47:52.445 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-06-08T03:47:52.471 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-08T03:47:52.496 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-06-08T03:47:52.530 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-06-08T03:47:52.555 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-06-08T03:47:52.605 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.8 MB/s | 138 kB 00:00 2023-06-08T03:47:52.648 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-06-08T03:47:52.689 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-06-08T03:47:52.732 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-06-08T03:47:52.773 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-06-08T03:47:52.815 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 931 kB/s | 39 kB 00:00 2023-06-08T03:47:52.903 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 7.7 MB/s | 688 kB 00:00 2023-06-08T03:47:52.957 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 9.7 MB/s | 538 kB 00:00 2023-06-08T03:47:52.999 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-06-08T03:47:53.141 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): python3-scipy-1.0.0-21.module+el8.5. 23 MB/s | 14 MB 00:00 2023-06-08T03:47:53.190 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 985 kB/s | 187 kB 00:00 2023-06-08T03:47:53.233 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-06-08T03:47:53.275 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 636 kB/s | 27 kB 00:00 2023-06-08T03:47:53.317 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-08T03:47:53.359 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-06-08T03:47:53.435 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-06-08T03:47:53.502 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-06-08T03:47:53.544 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-06-08T03:47:53.602 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-06-08T03:47:53.627 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 766 kB/s | 64 kB 00:00 2023-06-08T03:47:53.653 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-06-08T03:47:53.678 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-06-08T03:47:53.695 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-06-08T03:47:53.729 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-06-08T03:47:53.796 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-06-08T03:47:53.888 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 25 MB/s | 4.9 MB 00:00 2023-06-08T03:47:53.930 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-06-08T03:47:54.005 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-06-08T03:47:54.030 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.5 MB/s | 354 kB 00:00 2023-06-08T03:47:54.089 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-06-08T03:47:54.172 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): postgresql-server-10.21-2.module+el8 30 MB/s | 5.1 MB 00:00 2023-06-08T03:47:54.223 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-06-08T03:47:55.953 INFO:teuthology.orchestra.run.smithi079.stdout:(131/131): ceph-test-18.0.0-4313.ga02638f8.el8. 980 kB/s | 41 MB 00:42 2023-06-08T03:47:55.958 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-06-08T03:47:55.958 INFO:teuthology.orchestra.run.smithi079.stdout:Total 3.6 MB/s | 197 MB 00:54 2023-06-08T03:47:56.414 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): librados2-18.0.0-4313.ga02638f8.el8. 1.6 MB/s | 3.7 MB 00:02 2023-06-08T03:47:56.465 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-06-08T03:47:56.742 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-06-08T03:47:56.862 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-06-08T03:47:56.862 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-06-08T03:47:57.975 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-06-08T03:47:57.979 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-06-08T03:47:58.265 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librbd1-18.0.0-4313.ga02638f8.el8.x8 902 kB/s | 3.6 MB 00:04 2023-06-08T03:48:01.006 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): ceph-test-18.0.0-4313.ga02638f8.el8. 1.2 MB/s | 41 MB 00:33 2023-06-08T03:48:01.009 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-06-08T03:48:01.009 INFO:teuthology.orchestra.run.smithi156.stdout:Total 4.3 MB/s | 197 MB 00:45 2023-06-08T03:48:01.832 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-06-08T03:48:01.953 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-06-08T03:48:01.953 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-06-08T03:48:02.764 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-06-08T03:48:02.915 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-08T03:48:03.041 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-06-08T03:48:03.046 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-06-08T03:48:03.098 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-08T03:48:03.284 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-08T03:48:03.426 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-08T03:48:03.667 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-08T03:48:04.070 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-08T03:48:04.292 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 5/134 2023-06-08T03:48:04.673 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 5/134 2023-06-08T03:48:04.884 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-08T03:48:05.011 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-08T03:48:05.183 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-08T03:48:05.282 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-08T03:48:05.455 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-08T03:48:05.658 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-08T03:48:05.911 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-08T03:48:06.136 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-08T03:48:06.555 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-08T03:48:06.770 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-08T03:48:06.950 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-08T03:48:07.056 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-08T03:48:07.208 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-08T03:48:07.382 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-08T03:48:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-08T03:48:07.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-08T03:48:07.946 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-08T03:48:08.082 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-08T03:48:08.096 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-06-08T03:48:08.215 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-08T03:48:08.279 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-08T03:48:08.383 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-08T03:48:08.504 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-08T03:48:08.648 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-08T03:48:08.796 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-08T03:48:09.014 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-08T03:48:09.015 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-08T03:48:09.199 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-08T03:48:09.409 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-08T03:48:09.689 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 5/134 2023-06-08T03:48:09.780 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-08T03:48:09.949 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-08T03:48:10.085 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 5/134 2023-06-08T03:48:10.347 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-08T03:48:10.472 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-08T03:48:10.630 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-08T03:48:10.688 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-08T03:48:10.828 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-08T03:48:10.994 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-08T03:48:11.096 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-08T03:48:11.189 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-08T03:48:11.366 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-08T03:48:11.450 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-08T03:48:11.593 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-08T03:48:11.700 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-08T03:48:11.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 26/134 2023-06-08T03:48:11.902 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 27/134 2023-06-08T03:48:12.111 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-08T03:48:12.148 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 28/134 2023-06-08T03:48:12.324 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-08T03:48:12.376 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-08T03:48:12.572 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-08T03:48:12.693 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 14/134 2023-06-08T03:48:12.888 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-08T03:48:13.036 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 15/134 2023-06-08T03:48:13.259 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-08T03:48:13.444 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-08T03:48:13.660 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-08T03:48:13.821 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-08T03:48:13.946 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-08T03:48:14.073 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-08T03:48:14.121 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-08T03:48:14.733 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-08T03:48:14.955 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-08T03:48:15.095 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-08T03:48:15.412 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-08T03:48:15.550 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-08T03:48:15.650 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-08T03:48:15.754 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-08T03:48:15.872 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-08T03:48:16.067 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-08T03:48:16.241 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-08T03:48:16.420 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 36/134 2023-06-08T03:48:16.504 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-08T03:48:16.554 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 37/134 2023-06-08T03:48:16.696 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-08T03:48:16.794 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-08T03:48:16.840 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-08T03:48:16.977 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-08T03:48:17.042 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-08T03:48:17.114 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-08T03:48:17.181 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-08T03:48:17.338 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-08T03:48:17.361 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 26/134 2023-06-08T03:48:17.482 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 27/134 2023-06-08T03:48:17.542 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-08T03:48:17.697 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-08T03:48:17.751 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 28/134 2023-06-08T03:48:17.767 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-08T03:48:17.971 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-08T03:48:17.984 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-08T03:48:18.134 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-08T03:48:18.323 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-08T03:48:18.502 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-08T03:48:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-08T03:48:18.833 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-08T03:48:19.034 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-08T03:48:19.249 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-08T03:48:19.612 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-08T03:48:19.735 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-08T03:48:19.865 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-08T03:48:20.075 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-08T03:48:20.301 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-08T03:48:20.526 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-08T03:48:20.769 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-08T03:48:20.888 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-08T03:48:21.076 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-08T03:48:21.129 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-08T03:48:21.323 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-08T03:48:21.348 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-08T03:48:21.576 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-08T03:48:21.768 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-08T03:48:21.860 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-08T03:48:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-08T03:48:22.108 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-08T03:48:22.262 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-08T03:48:22.323 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-08T03:48:22.509 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-08T03:48:22.542 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 36/134 2023-06-08T03:48:22.685 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 37/134 2023-06-08T03:48:22.759 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-08T03:48:22.869 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-08T03:48:23.021 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-08T03:48:23.030 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-08T03:48:23.220 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-08T03:48:23.221 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-08T03:48:23.278 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-08T03:48:23.415 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-08T03:48:23.548 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-08T03:48:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-08T03:48:23.774 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-08T03:48:23.886 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-08T03:48:23.945 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-08T03:48:24.042 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-08T03:48:24.048 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-08T03:48:24.190 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-08T03:48:24.259 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-08T03:48:24.404 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-08T03:48:24.449 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-08T03:48:24.607 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-08T03:48:24.641 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-08T03:48:24.781 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-08T03:48:24.831 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-08T03:48:24.940 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-08T03:48:25.022 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-08T03:48:25.077 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-08T03:48:25.190 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-08T03:48:25.319 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-08T03:48:25.393 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-08T03:48:25.533 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-08T03:48:25.643 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-08T03:48:25.735 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-08T03:48:26.013 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-08T03:48:26.046 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-08T03:48:26.237 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-08T03:48:26.388 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-08T03:48:26.412 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-08T03:48:26.581 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-08T03:48:26.619 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-08T03:48:26.846 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-08T03:48:27.135 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-08T03:48:27.475 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-08T03:48:27.736 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-08T03:48:27.745 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-08T03:48:27.961 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-08T03:48:28.040 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-08T03:48:28.197 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-08T03:48:28.294 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-08T03:48:28.375 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-08T03:48:28.585 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-08T03:48:28.589 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-08T03:48:28.865 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-08T03:48:28.925 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-08T03:48:29.034 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-08T03:48:29.235 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-08T03:48:29.482 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-08T03:48:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-08T03:48:29.823 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-08T03:48:29.885 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-08T03:48:30.003 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-08T03:48:30.063 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-08T03:48:30.088 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-08T03:48:30.245 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 93/134 2023-06-08T03:48:30.371 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-08T03:48:30.617 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-08T03:48:30.773 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-08T03:48:30.962 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-08T03:48:31.186 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-08T03:48:31.380 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-08T03:48:31.597 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-08T03:48:31.797 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-08T03:48:31.917 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-08T03:48:32.210 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-08T03:48:32.448 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-08T03:48:32.616 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-08T03:48:32.892 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-08T03:48:33.126 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-08T03:48:33.309 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-08T03:48:33.514 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-08T03:48:33.784 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-08T03:48:33.995 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-08T03:48:34.575 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-08T03:48:34.619 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-08T03:48:34.859 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-08T03:48:34.880 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-08T03:48:34.939 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-08T03:48:35.093 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-08T03:48:35.118 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-08T03:48:35.393 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-08T03:48:35.678 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-08T03:48:35.856 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-08T03:48:36.663 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-08T03:48:36.843 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-08T03:48:36.905 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 92/134 2023-06-08T03:48:37.060 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 93/134 2023-06-08T03:48:40.722 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-08T03:48:40.968 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-08T03:48:41.555 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 94/134 2023-06-08T03:48:41.855 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-08T03:48:41.940 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 95/134 2023-06-08T03:48:42.125 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-08T03:48:53.664 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-08T03:48:53.664 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /run 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2023-06-08T03:48:53.665 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2023-06-08T03:48:53.666 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /dev 2023-06-08T03:48:53.666 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:48:54.040 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-08T03:48:55.616 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-08T03:48:55.788 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-08T03:48:56.006 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-08T03:48:56.156 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-08T03:48:56.604 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-08T03:48:56.806 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-08T03:48:56.822 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-08T03:48:58.423 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-08T03:48:58.623 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-08T03:48:58.744 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 102/134 2023-06-08T03:48:58.961 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 103/134 2023-06-08T03:48:59.155 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-08T03:48:59.370 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-08T03:48:59.686 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-08T03:49:00.646 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-08T03:49:00.813 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-08T03:49:00.870 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 96/134 2023-06-08T03:49:00.870 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-06-08T03:49:00.871 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-06-08T03:49:00.871 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-06-08T03:49:00.871 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-06-08T03:49:00.871 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-06-08T03:49:00.872 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-06-08T03:49:00.872 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-06-08T03:49:00.872 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-06-08T03:49:00.873 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-06-08T03:49:00.873 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:00.902 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-08T03:49:01.239 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-08T03:49:01.256 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 107/134 2023-06-08T03:49:01.486 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-08T03:49:01.521 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-08T03:49:01.688 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-08T03:49:01.699 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-08T03:49:02.822 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 97/134 2023-06-08T03:49:02.934 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 110/134 2023-06-08T03:49:02.977 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-08T03:49:03.150 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-08T03:49:03.184 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-08T03:49:03.195 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 98/134 2023-06-08T03:49:03.346 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-08T03:49:03.356 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-08T03:49:03.388 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-08T03:49:03.554 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-08T03:49:03.806 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 99/134 2023-06-08T03:49:03.899 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-08T03:49:04.005 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-08T03:49:04.023 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 100/134 2023-06-08T03:49:04.071 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-08T03:49:05.751 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-08T03:49:05.937 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 101/134 2023-06-08T03:49:06.086 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 102/134 2023-06-08T03:49:06.261 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 103/134 2023-06-08T03:49:06.471 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-08T03:49:06.686 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 104/134 2023-06-08T03:49:07.034 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-08T03:49:07.995 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 105/134 2023-06-08T03:49:07.997 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-08T03:49:08.179 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-08T03:49:08.272 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 106/134 2023-06-08T03:49:08.654 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 107/134 2023-06-08T03:49:08.864 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 115/134 2023-06-08T03:49:08.918 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-08T03:49:08.953 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 108/134 2023-06-08T03:49:09.129 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-08T03:49:09.140 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 109/134 2023-06-08T03:49:09.167 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-08T03:49:09.349 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 117/134 2023-06-08T03:49:09.588 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-08T03:49:09.736 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-08T03:49:09.925 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-08T03:49:10.165 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-08T03:49:10.306 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 121/134 2023-06-08T03:49:10.392 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 110/134 2023-06-08T03:49:10.445 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-08T03:49:10.647 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-08T03:49:10.681 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 111/134 2023-06-08T03:49:10.846 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-08T03:49:10.888 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4313.ga0263 112/134 2023-06-08T03:49:11.021 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-08T03:49:11.367 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 113/134 2023-06-08T03:49:11.504 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-08T03:49:12.498 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-08T03:49:12.719 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-08T03:49:12.737 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-08T03:49:12.894 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 124/134 2023-06-08T03:49:13.143 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 125/134 2023-06-08T03:49:13.429 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 126/134 2023-06-08T03:49:13.846 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-08T03:49:14.035 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-08T03:49:14.114 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-08T03:49:14.335 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-08T03:49:14.347 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-08T03:49:14.493 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-08T03:49:14.501 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-08T03:49:14.628 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-08T03:49:14.629 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-08T03:49:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-08T03:49:14.855 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-08T03:49:15.074 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-08T03:49:15.275 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-08T03:49:15.427 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 114/134 2023-06-08T03:49:15.541 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-08T03:49:15.541 INFO:teuthology.orchestra.run.smithi079.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-08T03:49:15.542 INFO:teuthology.orchestra.run.smithi079.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-08T03:49:15.542 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:15.753 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 1/134 2023-06-08T03:49:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2/134 2023-06-08T03:49:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 3/134 2023-06-08T03:49:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 4/134 2023-06-08T03:49:15.754 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 5/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 9/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 10/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 11/134 2023-06-08T03:49:15.755 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 12/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 13/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 14/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 15/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 16/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 17/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 18/134 2023-06-08T03:49:15.756 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 19/134 2023-06-08T03:49:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 20/134 2023-06-08T03:49:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 21/134 2023-06-08T03:49:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 22/134 2023-06-08T03:49:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 23/134 2023-06-08T03:49:15.757 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 24/134 2023-06-08T03:49:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 25/134 2023-06-08T03:49:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 26/134 2023-06-08T03:49:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 27/134 2023-06-08T03:49:15.765 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 28/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 29/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 30/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 31/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 32/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 33/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 34/134 2023-06-08T03:49:15.766 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 35/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 36/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 37/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 38/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-08T03:49:15.767 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-08T03:49:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-08T03:49:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-08T03:49:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-08T03:49:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-08T03:49:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-08T03:49:15.768 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-08T03:49:15.769 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-08T03:49:15.770 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-08T03:49:15.771 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-08T03:49:15.772 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-08T03:49:15.773 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-08T03:49:15.774 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-08T03:49:15.775 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-08T03:49:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-08T03:49:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-08T03:49:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-08T03:49:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-08T03:49:15.776 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-08T03:49:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-08T03:49:15.777 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-08T03:49:15.778 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-08T03:49:15.779 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 129/134 2023-06-08T03:49:15.780 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-08T03:49:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 131/134 2023-06-08T03:49:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-08T03:49:15.781 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-08T03:49:16.335 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 115/134 2023-06-08T03:49:16.710 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-08T03:49:16.931 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 117/134 2023-06-08T03:49:17.181 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-08T03:49:17.336 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-08T03:49:17.527 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-08T03:49:17.782 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-08T03:49:17.940 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 121/134 2023-06-08T03:49:18.104 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-08T03:49:18.516 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-08T03:49:18.516 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:02:29 ago on Thu 08 Jun 2023 03:46:49 AM UTC. 2023-06-08T03:49:20.326 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-08T03:49:20.586 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-08T03:49:20.603 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-08T03:49:20.805 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 124/134 2023-06-08T03:49:20.990 INFO:teuthology.orchestra.run.smithi079.stdout:Installed products updated. 2023-06-08T03:49:21.077 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 125/134 2023-06-08T03:49:21.208 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:21.208 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2023-06-08T03:49:21.208 INFO:teuthology.orchestra.run.smithi079.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.1-7.el8.noarch 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.0.4-10.el8.x86_64 2023-06-08T03:49:21.209 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.210 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.211 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:21.212 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.1-9.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: jq-1.6-3.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-08T03:49:21.213 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libpq-13.5-1.el8.x86_64 2023-06-08T03:49:21.214 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-08T03:49:21.215 INFO:teuthology.orchestra.run.smithi079.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-08T03:49:21.216 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-08T03:49:21.217 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.218 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-08T03:49:21.219 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-08T03:49:21.220 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-08T03:49:21.221 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-2.6-1.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-08T03:49:21.222 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-08T03:49:21.223 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-08T03:49:21.224 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-08T03:49:21.225 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-08T03:49:21.226 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-08T03:49:21.226 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-08T03:49:21.226 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-08T03:49:21.226 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-08T03:49:21.226 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:21.227 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-06-08T03:49:21.348 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 126/134 2023-06-08T03:49:21.406 DEBUG:teuthology.parallel:result is None 2023-06-08T03:49:21.778 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-08T03:49:21.986 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-08T03:49:22.072 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-08T03:49:22.319 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-08T03:49:22.332 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-08T03:49:22.527 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-08T03:49:22.535 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-08T03:49:22.679 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-08T03:49:22.679 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-08T03:49:22.905 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-08T03:49:22.905 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-08T03:49:23.184 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-08T03:49:23.360 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-08T03:49:23.684 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-08T03:49:23.685 INFO:teuthology.orchestra.run.smithi156.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-08T03:49:23.685 INFO:teuthology.orchestra.run.smithi156.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-08T03:49:23.685 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:23.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 1/134 2023-06-08T03:49:23.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2/134 2023-06-08T03:49:23.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 3/134 2023-06-08T03:49:23.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 4/134 2023-06-08T03:49:23.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4313.ga0263 5/134 2023-06-08T03:49:23.903 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 6/134 2023-06-08T03:49:23.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 7/134 2023-06-08T03:49:23.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 8/134 2023-06-08T03:49:23.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 9/134 2023-06-08T03:49:23.904 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 10/134 2023-06-08T03:49:23.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 11/134 2023-06-08T03:49:23.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 12/134 2023-06-08T03:49:23.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 13/134 2023-06-08T03:49:23.905 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_ 14/134 2023-06-08T03:49:23.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 15/134 2023-06-08T03:49:23.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 16/134 2023-06-08T03:49:23.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_6 17/134 2023-06-08T03:49:23.906 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86 18/134 2023-06-08T03:49:23.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 19/134 2023-06-08T03:49:23.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el 20/134 2023-06-08T03:49:23.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-4313.ga02638f8.el8. 21/134 2023-06-08T03:49:23.907 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_6 22/134 2023-06-08T03:49:23.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 23/134 2023-06-08T03:49:23.908 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 24/134 2023-06-08T03:49:23.916 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 25/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 26/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 27/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 28/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8. 29/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noa 30/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.n 31/134 2023-06-08T03:49:23.917 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02 32/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el 33/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 34/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.e 35/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 36/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 37/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 38/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-08T03:49:23.918 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-08T03:49:23.919 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-08T03:49:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-08T03:49:23.921 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-08T03:49:23.921 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-08T03:49:23.921 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-08T03:49:23.921 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-08T03:49:23.921 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-08T03:49:23.921 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-08T03:49:23.922 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-08T03:49:23.923 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-08T03:49:23.924 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-08T03:49:23.925 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-08T03:49:23.926 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-08T03:49:23.927 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-08T03:49:23.928 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-08T03:49:23.929 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 129/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-08T03:49:23.930 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 131/134 2023-06-08T03:49:23.931 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-08T03:49:23.931 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-08T03:49:26.831 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-08T03:49:26.832 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:02:23 ago on Thu 08 Jun 2023 03:47:03 AM UTC. 2023-06-08T03:49:29.482 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-06-08T03:49:29.710 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:29.710 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-06-08T03:49:29.710 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-08T03:49:29.710 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.710 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.711 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.712 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.713 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-4313.ga02638f8.el8.noarch 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-06-08T03:49:29.714 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.715 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-08T03:49:29.716 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-08T03:49:29.717 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-08T03:49:29.718 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-08T03:49:29.719 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-08T03:49:29.720 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-08T03:49:29.721 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-08T03:49:29.722 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-08T03:49:29.723 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-08T03:49:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-08T03:49:29.725 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.726 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.727 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-4313.ga02638f8.el8.x86_64 2023-06-08T03:49:29.727 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-08T03:49:29.727 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-08T03:49:29.727 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-08T03:49:29.727 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-08T03:49:29.727 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-08T03:49:29.728 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-08T03:49:29.728 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:29.728 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-06-08T03:49:29.933 DEBUG:teuthology.parallel:result is None 2023-06-08T03:49:29.933 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-08T03:49:30.144 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-08T03:49:30.172 INFO:teuthology.orchestra.run.smithi079.stdout:18.0.0-4313.ga02638f8.el8 2023-06-08T03:49:30.173 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4313.ga02638f8.el8 2023-06-08T03:49:30.173 INFO:teuthology.task.install:The correct ceph version 18.0.0-4313.ga02638f8 is installed. 2023-06-08T03:49:30.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a02638f808b8fe11bccdd01222c0e6e2eb71120b 2023-06-08T03:49:30.388 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-08T03:49:30.412 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-4313.ga02638f8.el8 2023-06-08T03:49:30.412 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4313.ga02638f8.el8 2023-06-08T03:49:30.413 INFO:teuthology.task.install:The correct ceph version 18.0.0-4313.ga02638f8 is installed. 2023-06-08T03:49:30.413 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-08T03:49:30.414 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:30.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-08T03:49:30.460 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:30.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-08T03:49:30.501 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-08T03:49:30.501 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:30.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2023-06-08T03:49:30.530 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-08T03:49:30.599 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:30.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-06-08T03:49:30.626 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-08T03:49:30.692 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-08T03:49:30.693 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:30.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-08T03:49:30.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-08T03:49:30.788 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:30.789 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-08T03:49:30.815 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-08T03:49:30.881 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-08T03:49:30.881 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:30.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2023-06-08T03:49:30.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-08T03:49:30.975 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:30.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-06-08T03:49:31.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-08T03:49:31.068 INFO:teuthology.run_tasks:Running task ceph... 2023-06-08T03:49:31.141 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-08T03:49:31.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2023-06-08T03:49:31.144 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /var/log/ceph 2023-06-08T03:49:31.171 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-08T03:49:31.171 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-08T03:49:31.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-08T03:49:31.238 INFO:tasks.ceph:Creating extra log directories... 2023-06-08T03:49:31.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-08T03:49:31.279 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-08T03:49:31.312 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-08T03:49:31.313 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-08T03:49:31.313 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-08T03:49:31.314 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:31.314 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-08T03:49:31.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-08T03:49:31.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-08T03:49:31.513 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:31.513 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-08T03:49:31.530 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-08T03:49:31.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-08T03:49:31.693 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-08T03:49:31.694 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-08T03:49:31.694 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi/7298112', 'branch': 'wip-lxb-fscrypt-20230607-0901', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7298112', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'xiubli-2023-06-08_03:32:08-fs:functional-wip-lxb-fscrypt-20230607-0901-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-lxb-fscrypt-20230607-0901'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lxb-fscrypt-20230607-0901', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}}, 'owner': 'scheduled_xiubli@teuthology', 'priority': 110, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-lxb-fscrypt-20230607-0901', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a02638f808b8fe11bccdd01222c0e6e2eb71120b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi079.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi156.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a02638f808b8fe11bccdd01222c0e6e2eb71120b'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61', 'timestamp': '2023-06-08_03:32:08', 'tube': 'smithi', 'user': 'xiubli', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-08T03:49:31.695 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-08T03:49:31.697 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-08T03:49:31.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2023-06-08T03:49:31.753 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/run/ceph 2023-06-08T03:49:31.810 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:31.811 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2023-06-08T03:49:31.834 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-08T03:49:31.834 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2023-06-08T03:49:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-08T03:49:31.890 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 374148 Links: 1 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-08 03:49:15.459713211 +0000 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-08 03:46:01.644330784 +0000 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-08 03:46:01.644330784 +0000 2023-06-08T03:49:31.891 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-08T03:49:31.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-08T03:49:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-08T03:49:31.955 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-08T03:49:31.956 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000266327 s, 1.9 MB/s 2023-06-08T03:49:31.956 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-08T03:49:32.014 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2023-06-08T03:49:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-08T03:49:32.070 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:32.071 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 364499 Links: 1 2023-06-08T03:49:32.071 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:32.071 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:32.071 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-08 03:49:15.459713211 +0000 2023-06-08T03:49:32.071 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-08 03:46:02.189317299 +0000 2023-06-08T03:49:32.071 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-08 03:46:02.189317299 +0000 2023-06-08T03:49:32.072 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-08T03:49:32.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-08T03:49:32.135 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-08T03:49:32.135 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-08T03:49:32.136 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000281034 s, 1.8 MB/s 2023-06-08T03:49:32.136 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-08T03:49:32.194 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2023-06-08T03:49:32.251 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-08T03:49:32.252 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 373174 Links: 1 2023-06-08T03:49:32.252 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:32.253 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:32.253 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-08 03:49:15.459713211 +0000 2023-06-08T03:49:32.253 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-08 03:46:02.714304310 +0000 2023-06-08T03:49:32.253 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-08 03:46:02.714304310 +0000 2023-06-08T03:49:32.254 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-08T03:49:32.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-08T03:49:32.316 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-08T03:49:32.317 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-08T03:49:32.317 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000225809 s, 2.3 MB/s 2023-06-08T03:49:32.318 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-08T03:49:32.375 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2023-06-08T03:49:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-08T03:49:32.432 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:32.432 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 6h/6d Inode: 377866 Links: 1 2023-06-08T03:49:32.432 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:32.433 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:32.433 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2023-06-08 03:49:15.459713211 +0000 2023-06-08T03:49:32.433 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2023-06-08 03:46:03.247291123 +0000 2023-06-08T03:49:32.434 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2023-06-08 03:46:03.247291123 +0000 2023-06-08T03:49:32.434 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: - 2023-06-08T03:49:32.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-08T03:49:32.496 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2023-06-08T03:49:32.496 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2023-06-08T03:49:32.496 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000174655 s, 2.9 MB/s 2023-06-08T03:49:32.497 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-08T03:49:32.554 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:32.555 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:32.555 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-06-08T03:49:32.574 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-08T03:49:32.574 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-06-08T03:49:32.632 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-08T03:49:32.633 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:32.633 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 374810 Links: 1 2023-06-08T03:49:32.633 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:32.633 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:32.633 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-06-08 03:49:23.589994141 +0000 2023-06-08T03:49:32.633 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-06-08 03:45:33.187874672 +0000 2023-06-08T03:49:32.634 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-06-08 03:45:33.187874672 +0000 2023-06-08T03:49:32.634 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-06-08T03:49:32.634 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-08T03:49:32.700 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-06-08T03:49:32.700 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-06-08T03:49:32.701 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000133626 s, 3.8 MB/s 2023-06-08T03:49:32.701 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-08T03:49:32.761 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-06-08T03:49:32.818 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-08T03:49:32.819 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:32.819 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 363034 Links: 1 2023-06-08T03:49:32.819 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:32.819 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:32.820 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-06-08 03:49:23.589994141 +0000 2023-06-08T03:49:32.820 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-06-08 03:45:33.733866536 +0000 2023-06-08T03:49:32.820 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-06-08 03:45:33.733866536 +0000 2023-06-08T03:49:32.820 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-06-08T03:49:32.821 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-08T03:49:32.885 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-06-08T03:49:32.885 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-06-08T03:49:32.885 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 9.9783e-05 s, 5.1 MB/s 2023-06-08T03:49:32.886 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-08T03:49:32.944 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-06-08T03:49:33.002 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-08T03:49:33.002 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:33.002 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 373384 Links: 1 2023-06-08T03:49:33.003 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:33.003 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:33.003 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-06-08 03:49:23.589994141 +0000 2023-06-08T03:49:33.004 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-06-08 03:45:34.276858444 +0000 2023-06-08T03:49:33.004 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-06-08 03:45:34.276858444 +0000 2023-06-08T03:49:33.004 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-06-08T03:49:33.005 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-08T03:49:33.066 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-06-08T03:49:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-06-08T03:49:33.067 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000107552 s, 4.8 MB/s 2023-06-08T03:49:33.068 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-08T03:49:33.125 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-06-08T03:49:33.182 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-08T03:49:33.183 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-08T03:49:33.183 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 375091 Links: 1 2023-06-08T03:49:33.183 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-08T03:49:33.184 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-06-08T03:49:33.184 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-06-08 03:49:23.589994141 +0000 2023-06-08T03:49:33.184 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-06-08 03:45:34.793850740 +0000 2023-06-08T03:49:33.184 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-06-08 03:45:34.793850740 +0000 2023-06-08T03:49:33.185 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-06-08T03:49:33.185 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-08T03:49:33.246 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-06-08T03:49:33.247 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-06-08T03:49:33.247 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000117428 s, 4.4 MB/s 2023-06-08T03:49:33.248 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-08T03:49:33.305 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:33.305 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-08T03:49:33.306 INFO:tasks.ceph:Generating config... 2023-06-08T03:49:33.307 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-08T03:49:33.307 INFO:tasks.ceph:[client] debug client = 20 2023-06-08T03:49:33.307 INFO:tasks.ceph:[client] debug ms = 1 2023-06-08T03:49:33.308 INFO:tasks.ceph:[client] pid file = 2023-06-08T03:49:33.308 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-08T03:49:33.309 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-08T03:49:33.309 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-08T03:49:33.309 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-06-08T03:49:33.310 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-08T03:49:33.310 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-08T03:49:33.310 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-08T03:49:33.310 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-08T03:49:33.311 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-08T03:49:33.311 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-08T03:49:33.311 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-08T03:49:33.312 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-08T03:49:33.312 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-08T03:49:33.312 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-08T03:49:33.312 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-08T03:49:33.313 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-08T03:49:33.313 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-08T03:49:33.313 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-08T03:49:33.313 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-08T03:49:33.313 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-08T03:49:33.314 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-06-08T03:49:33.314 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-06-08T03:49:33.314 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-08T03:49:33.314 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-08T03:49:33.315 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-06-08T03:49:33.315 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-08T03:49:33.315 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-08T03:49:33.316 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-08T03:49:33.316 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-08T03:49:33.316 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-08T03:49:33.316 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-08T03:49:33.317 INFO:tasks.ceph:Setting up mon.a... 2023-06-08T03:49:33.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-08T03:49:33.375 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.keyring 2023-06-08T03:49:33.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-08T03:49:33.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-08T03:49:33.452 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.79'), ('mon.b', '172.21.15.156'), ('mon.c', '[v2:172.21.15.156:3301,v1:172.21.15.156:6790]')] 2023-06-08T03:49:33.452 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.79,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-08T03:49:33.453 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:33.454 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-08T03:49:33.508 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.79 --add b 172.21.15.156 --addv c '[v2:172.21.15.156:3301,v1:172.21.15.156:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:33.596 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:33.597 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: generated fsid 141881e0-5474-41d9-8205-1a44c31c5e0b 2023-06-08T03:49:33.597 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = pacific 2023-06-08T03:49:33.598 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 0 2023-06-08T03:49:33.598 INFO:teuthology.orchestra.run.smithi079.stdout:fsid 141881e0-5474-41d9-8205-1a44c31c5e0b 2023-06-08T03:49:33.598 INFO:teuthology.orchestra.run.smithi079.stdout:last_changed 2023-06-08T03:49:33.594151+0000 2023-06-08T03:49:33.598 INFO:teuthology.orchestra.run.smithi079.stdout:created 2023-06-08T03:49:33.594151+0000 2023-06-08T03:49:33.599 INFO:teuthology.orchestra.run.smithi079.stdout:min_mon_release 16 (pacific) 2023-06-08T03:49:33.599 INFO:teuthology.orchestra.run.smithi079.stdout:election_strategy: 1 2023-06-08T03:49:33.599 INFO:teuthology.orchestra.run.smithi079.stdout:0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2023-06-08T03:49:33.599 INFO:teuthology.orchestra.run.smithi079.stdout:1: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2023-06-08T03:49:33.599 INFO:teuthology.orchestra.run.smithi079.stdout:2: [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] mon.c 2023-06-08T03:49:33.600 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-08T03:49:33.601 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-08T03:49:33.617 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 141881e0-5474-41d9-8205-1a44c31c5e0b... 2023-06-08T03:49:33.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-08T03:49:33.659 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-08T03:49:33.701 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2023-06-08T03:49:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2023-06-08T03:49:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-08T03:49:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2023-06-08T03:49:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.702 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2023-06-08T03:49:33.703 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.704 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2023-06-08T03:49:33.705 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.706 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.79,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2023-06-08T03:49:33.707 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = 141881e0-5474-41d9-8205-1a44c31c5e0b 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2023-06-08T03:49:33.708 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.709 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2023-06-08T03:49:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2023-06-08T03:49:33.711 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 20 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 20 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 10 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2023-06-08T03:49:33.712 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2023-06-08T03:49:33.713 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2023-06-08T03:49:33.714 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2023-06-08T03:49:33.715 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-08T03:49:33.716 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: mon op complaint time = 120 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2023-06-08T03:49:33.717 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: client mount timeout = 600 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: debug client = 20 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-08T03:49:33.718 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = "" 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 900 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 900 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c] 2023-06-08T03:49:33.719 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d] 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout:[mds] 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds = 20 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds balancer = 20 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug frag = True 2023-06-08T03:49:33.720 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug scatterstat = True 2023-06-08T03:49:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: mds op complaint time = 180 2023-06-08T03:49:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: mds verify scatter = True 2023-06-08T03:49:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2023-06-08T03:49:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 900 2023-06-08T03:49:33.721 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 900 2023-06-08T03:49:33.722 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2023-06-08T03:49:33.722 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2023-06-08T03:49:33.723 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-08T03:49:33.724 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.725 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-08T03:49:33.726 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2023-06-08T03:49:33.727 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.79,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = 141881e0-5474-41d9-8205-1a44c31c5e0b 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2023-06-08T03:49:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2023-06-08T03:49:33.729 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.730 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2023-06-08T03:49:33.731 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-06-08T03:49:33.732 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-06-08T03:49:33.733 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-06-08T03:49:33.734 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2023-06-08T03:49:33.735 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-08T03:49:33.736 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2023-06-08T03:49:33.737 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2023-06-08T03:49:33.738 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-06-08T03:49:33.739 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-06-08T03:49:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2023-06-08T03:49:33.741 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-06-08T03:49:33.742 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-06-08T03:49:33.742 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-06-08T03:49:33.742 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-08T03:49:33.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-08T03:49:33.818 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-08T03:49:33.819 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:33.819 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-08T03:49:33.835 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:33.835 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-08T03:49:33.890 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:49:33.891 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:33.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-08T03:49:33.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-08T03:49:33.965 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:49:33.966 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:34.022 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:49:34.022 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:34.023 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-08T03:49:34.023 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-08T03:49:34.060 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:49:34.060 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:34.116 INFO:tasks.ceph:Setting up mon nodes... 2023-06-08T03:49:34.117 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-08T03:49:34.117 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-08T03:49:34.169 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-08T03:49:34.172 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-08T03:49:34.261 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-08T03:49:34.264 INFO:tasks.ceph:Setting up mds nodes... 2023-06-08T03:49:34.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-08T03:49:34.317 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-08T03:49:34.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-08T03:49:34.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-08T03:49:34.437 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-08T03:49:34.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-08T03:49:34.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-08T03:49:34.519 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-08T03:49:34.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-08T03:49:34.547 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-08T03:49:34.640 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-08T03:49:34.642 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-08T03:49:34.668 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-08T03:49:34.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-08T03:49:34.713 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-08T03:49:34.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-08T03:49:34.768 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-08T03:49:34.781 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-08T03:49:34.823 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-08T03:49:34.836 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-08T03:49:34.919 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-08T03:49:34.932 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-08T03:49:34.932 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-08T03:49:34.932 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-08T03:49:34.958 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:34.958 INFO:tasks.ceph:role: osd.0 2023-06-08T03:49:34.959 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:49:34.959 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-08T03:49:35.033 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:35.033 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:35.033 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:35.033 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:35.034 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:35.034 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:35.034 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:35.034 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:35.034 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:35.034 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:35.453 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-08T03:49:35.455 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-08T03:49:35.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-08T03:49:35.603 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-08T03:49:35.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-08T03:49:35.708 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:35.708 INFO:tasks.ceph:role: osd.1 2023-06-08T03:49:35.708 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:49:35.709 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-08T03:49:35.782 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:35.782 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:35.783 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:35.783 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:35.783 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:35.783 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:35.783 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:36.207 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-08T03:49:36.209 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-08T03:49:36.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-08T03:49:36.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-08T03:49:36.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-08T03:49:36.419 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:36.420 INFO:tasks.ceph:role: osd.2 2023-06-08T03:49:36.420 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:49:36.420 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-08T03:49:36.495 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:36.495 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:36.495 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:36.496 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:36.956 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-08T03:49:36.957 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-08T03:49:36.958 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-08T03:49:37.087 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-08T03:49:37.121 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-08T03:49:37.190 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:37.190 INFO:tasks.ceph:role: osd.3 2023-06-08T03:49:37.190 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:49:37.190 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-08T03:49:37.265 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:37.265 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:37.266 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:37.718 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2023-06-08T03:49:37.719 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2023-06-08T03:49:37.720 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-08T03:49:37.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-08T03:49:37.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:38.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:38.023+0000 7f0579324640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-08T03:49:38.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:38.023+0000 7f0579324640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-08T03:49:38.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:38.023+0000 7f0579324640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-08T03:49:38.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:38.023+0000 7f0579324640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-08T03:49:38.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:38.023+0000 7f0579324640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-08T03:49:38.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:38.023+0000 7f0579324640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-08T03:49:40.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-08T03:49:40.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:41.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:41.028+0000 7f98adf72640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-08T03:49:41.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:41.028+0000 7f98adf72640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-08T03:49:41.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:41.028+0000 7f98adf72640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-08T03:49:41.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:41.028+0000 7f98adf72640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-08T03:49:41.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:41.028+0000 7f98adf72640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-08T03:49:41.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:41.028+0000 7f98adf72640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-08T03:49:43.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-08T03:49:43.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:44.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:44.013+0000 7f8a64a18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-08T03:49:44.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:44.013+0000 7f8a64a18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-08T03:49:44.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:44.013+0000 7f8a64a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-08T03:49:44.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:44.013+0000 7f8a64a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-08T03:49:44.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:44.013+0000 7f8a64a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-08T03:49:44.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:44.014+0000 7f8a64a18640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-08T03:49:46.881 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-08T03:49:46.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:47.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:46.998+0000 7f024590e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-08T03:49:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:46.998+0000 7f024590e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-08T03:49:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:46.998+0000 7f024590e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-08T03:49:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:46.998+0000 7f024590e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-08T03:49:47.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:46.998+0000 7f024590e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-08T03:49:47.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:49:46.999+0000 7f024590e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-08T03:49:49.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-08T03:49:49.890 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-08T03:49:49.922 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:49.923 INFO:tasks.ceph:role: osd.4 2023-06-08T03:49:49.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:49:49.923 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-08T03:49:50.018 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:50.018 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:50.019 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:50.020 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:50.040 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-06-08T03:49:50.097 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-06-08T03:49:50.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-08T03:49:50.140 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-08T03:49:50.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-08T03:49:50.280 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:50.280 INFO:tasks.ceph:role: osd.5 2023-06-08T03:49:50.280 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:49:50.281 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-08T03:49:50.345 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:50.346 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:50.346 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:50.346 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:50.346 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:50.346 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:50.347 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:50.347 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:50.347 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:50.347 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:50.369 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-06-08T03:49:50.370 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-06-08T03:49:50.370 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-08T03:49:50.451 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-08T03:49:50.521 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-08T03:49:50.587 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:50.588 INFO:tasks.ceph:role: osd.6 2023-06-08T03:49:50.588 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:49:50.588 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-08T03:49:50.653 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:50.654 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:50.655 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:50.655 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:50.676 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-06-08T03:49:50.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-06-08T03:49:50.678 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-08T03:49:50.757 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-08T03:49:50.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-08T03:49:50.902 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-08T03:49:50.902 INFO:tasks.ceph:role: osd.7 2023-06-08T03:49:50.902 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:49:50.903 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-08T03:49:50.968 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-08T03:49:50.968 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-08T03:49:50.968 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-08T03:49:50.968 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-08T03:49:50.968 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-08T03:49:50.968 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-06-08T03:49:50.969 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-08T03:49:50.969 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-08T03:49:50.969 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-08T03:49:50.969 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-08T03:49:50.990 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-06-08T03:49:50.992 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-06-08T03:49:50.992 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-08T03:49:51.072 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-08T03:49:51.142 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:51.248 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:51.244+0000 7f10b6eab640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-08T03:49:51.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:51.245+0000 7f10b6eab640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-08T03:49:51.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:51.245+0000 7f10b6eab640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-08T03:49:51.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:51.245+0000 7f10b6eab640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-08T03:49:51.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:51.245+0000 7f10b6eab640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-08T03:49:51.249 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:51.245+0000 7f10b6eab640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-08T03:49:54.120 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-08T03:49:54.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:54.238+0000 7fd77844a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-08T03:49:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:54.238+0000 7fd77844a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-08T03:49:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:54.238+0000 7fd77844a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-08T03:49:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:54.238+0000 7fd77844a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-08T03:49:54.242 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:54.238+0000 7fd77844a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-08T03:49:54.243 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:54.239+0000 7fd77844a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-08T03:49:57.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-08T03:49:57.134 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:49:57.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:57.223+0000 7effaa0f8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-08T03:49:57.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:57.223+0000 7effaa0f8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-08T03:49:57.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:57.223+0000 7effaa0f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-08T03:49:57.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:57.223+0000 7effaa0f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-08T03:49:57.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:57.223+0000 7effaa0f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-08T03:49:57.227 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:49:57.223+0000 7effaa0f8640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-08T03:50:00.088 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-08T03:50:00.120 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:50:00.211 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:00.208+0000 7f72a52e7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-08T03:50:00.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:00.208+0000 7f72a52e7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-08T03:50:00.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:00.208+0000 7f72a52e7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-08T03:50:00.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:00.208+0000 7f72a52e7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-08T03:50:00.212 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:00.208+0000 7f72a52e7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-08T03:50:00.213 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:00.208+0000 7f72a52e7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-08T03:50:01.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:50:01.698 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:50:03.076 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-08T03:50:03.107 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-08T03:50:03.108 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-08T03:50:03.140 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-08T03:50:03.209 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-08T03:50:03.278 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-08T03:50:03.345 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-08T03:50:03.411 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-08T03:50:03.478 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-08T03:50:03.544 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.544 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-08T03:50:03.576 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.576 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-08T03:50:03.646 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.646 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-08T03:50:03.714 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.714 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-08T03:50:03.782 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.782 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-08T03:50:03.848 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.849 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-08T03:50:03.915 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:03.916 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-08T03:50:03.982 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.982 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-08T03:50:03.999 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:03.999 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-08T03:50:04.055 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:04.055 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-08T03:50:04.072 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:04.072 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-08T03:50:04.128 INFO:tasks.ceph:Adding keys to all mons... 2023-06-08T03:50:04.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-08T03:50:04.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-08T03:50:04.171 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.y] 2023-06-08T03:50:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FkDgb5CRAAzNxBhP5cwvsr6T+vPZgjbQ== 2023-06-08T03:50:04.171 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2023-06-08T03:50:04.171 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FkRZDBEhAAN5RLnYkm+w9jFxGSf8TjeQ== 2023-06-08T03:50:04.172 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c] 2023-06-08T03:50:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4Fk4cf2GRAAyRD7ooUmmArn9eNTBu9p1Q== 2023-06-08T03:50:04.172 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2023-06-08T03:50:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDST4FkTVJmARAAwhbt9gcKhJmxNMvLCCZNgA== 2023-06-08T03:50:04.172 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2023-06-08T03:50:04.172 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDVT4Fk1ICyARAARd9komdmzhtLzhWG1Cwa9g== 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDYT4FkjaLQABAAjXntGlmvUt4Eixrlsqv6pg== 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDaT4FkfZmZOxAAYEUB+u4uXJPWRr+iCIEzfw== 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4Fkc+5vDxAAFCxhBuMs5zg/ox2OeO3i8w== 2023-06-08T03:50:04.173 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FkRujWHhAAP6t5Gy6nJqvbJflDSdvM9Q== 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d] 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FkqmMJJhAAO3QcZ8jcPmU9RndPFbc2jg== 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDfT4FkLEajDhAAFaS4OndZz6gW+9pJinUziw== 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2023-06-08T03:50:04.174 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDiT4Fkh5tEDhAANLbWNGFTbafQ3h3FspueuA== 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDlT4Fk+GtZDRAAd72KpRrYlvYq5OHVbbrNbg== 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDoT4FkbWlyDBAAtLYK0i2yjivtP0+2XgRY9Q== 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FknZFgKhAA1Hf83RGBLm0IVJwZZeXrNg== 2023-06-08T03:50:04.175 INFO:teuthology.orchestra.run.smithi079.stdout:[client.2] 2023-06-08T03:50:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FkqH+rLRAAbpuUe2nkVYasrtpRL1jcIA== 2023-06-08T03:50:04.176 INFO:teuthology.orchestra.run.smithi079.stdout:[client.1] 2023-06-08T03:50:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4Fk31fyMBAAa2EcGDOxAywwlFcdTFwhPA== 2023-06-08T03:50:04.176 INFO:teuthology.orchestra.run.smithi079.stdout:[client.3] 2023-06-08T03:50:04.176 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDOT4FkXmCmNhAAOyOsxsYfveZPi1hP/apNjw== 2023-06-08T03:50:04.192 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.y] 2023-06-08T03:50:04.193 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FkDgb5CRAAzNxBhP5cwvsr6T+vPZgjbQ== 2023-06-08T03:50:04.193 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-06-08T03:50:04.193 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FkRZDBEhAAN5RLnYkm+w9jFxGSf8TjeQ== 2023-06-08T03:50:04.193 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-06-08T03:50:04.193 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4Fk4cf2GRAAyRD7ooUmmArn9eNTBu9p1Q== 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.0] 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDST4FkTVJmARAAwhbt9gcKhJmxNMvLCCZNgA== 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.1] 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDVT4Fk1ICyARAARd9komdmzhtLzhWG1Cwa9g== 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.2] 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDYT4FkjaLQABAAjXntGlmvUt4Eixrlsqv6pg== 2023-06-08T03:50:04.194 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.3] 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDaT4FkfZmZOxAAYEUB+u4uXJPWRr+iCIEzfw== 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.x] 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4Fkc+5vDxAAFCxhBuMs5zg/ox2OeO3i8w== 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FkRujWHhAAP6t5Gy6nJqvbJflDSdvM9Q== 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-06-08T03:50:04.195 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FkqmMJJhAAO3QcZ8jcPmU9RndPFbc2jg== 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.4] 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDfT4FkLEajDhAAFaS4OndZz6gW+9pJinUziw== 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.5] 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDiT4Fkh5tEDhAANLbWNGFTbafQ3h3FspueuA== 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.6] 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDlT4Fk+GtZDRAAd72KpRrYlvYq5OHVbbrNbg== 2023-06-08T03:50:04.196 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.7] 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDoT4FkbWlyDBAAtLYK0i2yjivtP0+2XgRY9Q== 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FknZFgKhAA1Hf83RGBLm0IVJwZZeXrNg== 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout:[client.2] 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FkqH+rLRAAbpuUe2nkVYasrtpRL1jcIA== 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout:[client.1] 2023-06-08T03:50:04.197 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4Fk31fyMBAAa2EcGDOxAywwlFcdTFwhPA== 2023-06-08T03:50:04.198 INFO:teuthology.orchestra.run.smithi156.stdout:[client.3] 2023-06-08T03:50:04.198 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDOT4FkXmCmNhAAOyOsxsYfveZPi1hP/apNjw== 2023-06-08T03:50:04.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-08T03:50:04.219 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-08T03:50:04.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.294 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.353 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.412 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.467 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.576 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-08T03:50:04.631 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-08T03:50:04.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.686 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.744 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-08T03:50:04.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.857 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:04.966 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-08T03:50:05.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.136 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.194 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-08T03:50:05.250 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-08T03:50:05.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-08T03:50:05.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-08T03:50:05.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-08T03:50:05.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-08T03:50:05.428 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-08T03:50:05.525 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-08T03:50:05.552 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-08T03:50:05.618 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-08T03:50:05.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-08T03:50:05.741 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:50:05.744 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-08T03:50:05.797 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-08T03:50:05.797 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-08T03:50:05.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-08T03:50:05.800 INFO:tasks.ceph.mon.a:Started 2023-06-08T03:50:05.800 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-08T03:50:05.800 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-08T03:50:05.839 INFO:tasks.ceph.mon.b:Started 2023-06-08T03:50:05.840 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-08T03:50:05.840 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-08T03:50:05.842 INFO:tasks.ceph.mon.c:Started 2023-06-08T03:50:05.842 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-08T03:50:05.843 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-08T03:50:05.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-08T03:50:05.845 INFO:tasks.ceph.mgr.y:Started 2023-06-08T03:50:05.845 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-08T03:50:05.846 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-08T03:50:05.847 INFO:tasks.ceph.mgr.x:Started 2023-06-08T03:50:05.848 INFO:tasks.ceph:Setting crush tunables to default 2023-06-08T03:50:05.848 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd crush tunables default 2023-06-08T03:50:05.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.966+0000 7f684e835700 1 Processor -- start 2023-06-08T03:50:05.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.966+0000 7f684e835700 1 -- start start 2023-06-08T03:50:05.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684e835700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:05.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684e835700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:05.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684e835700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6848071170 con 0x7f6848158780 2023-06-08T03:50:05.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684e835700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684806e110 con 0x7f68480ccb30 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684e835700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68480c9a90 con 0x7f68480707d0 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33748/0 (socket says 172.21.15.79:33748) 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f6847fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6848158780 0x7f68480c9380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57222/0 (socket says 172.21.15.79:57222) 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 -- 172.21.15.79:0/3848120297 learned_addr learned my addr 172.21.15.79:0/3848120297 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 msgr2=0x7f684806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-08T03:50:05.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 msgr2=0x7f684806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-08T03:50:05.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:05.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-08T03:50:05.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-08T03:50:05.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684cdd2700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 msgr2=0x7f6848070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-08T03:50:05.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684cdd2700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 msgr2=0x7f6848070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-08T03:50:05.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-08T03:50:05.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:05.967+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-08T03:50:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f68477fe700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 msgr2=0x7f684806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f68477fe700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 msgr2=0x7f684806b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f684cdd2700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 msgr2=0x7f6848070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f684cdd2700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 msgr2=0x7f6848070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-08T03:50:06.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-08T03:50:06.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.168+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-08T03:50:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265124158 0 0) 0x7f6848071170 con 0x7f6848158780 2023-06-08T03:50:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68300035b0 con 0x7f6848158780 2023-06-08T03:50:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 404590925 0 0) 0x7f68300035b0 con 0x7f6848158780 2023-06-08T03:50:06.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 msgr2=0x7f6848070c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.156:3300/0 conn(0x7f68480707d0 0x7f6848070c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 msgr2=0x7f684806b230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 --2- 172.21.15.79:0/3848120297 >> v2:172.21.15.79:3300/0 conn(0x7f68480ccb30 0x7f684806b230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684806da90 con 0x7f6848158780 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.170+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f6838001670 con 0x7f6848158780 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 >> v1:172.21.15.79:6789/0 conn(0x7f6848158780 legacy=0x7f68480c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6830003a30 0x7f6830003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830004850 0x7f6830004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830005200 0x7f683000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684806e110 con 0x7f6830003a30 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480c9a90 con 0x7f6830004850 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683000c190 con 0x7f6830005200 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6830003a30 0x7f6830003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830004850 0x7f6830004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f684cdd2700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830005200 0x7f683000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 -- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830005200 msgr2=0x7f683000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830005200 0x7f683000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 -- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830004850 msgr2=0x7f6830004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830004850 0x7f6830004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 -- 172.21.15.79:0/3848120297 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300035b0 con 0x7f6830003a30 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f684cdd2700 1 -- 172.21.15.79:0/3848120297 reap_dead start 2023-06-08T03:50:06.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f68477fe700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830004850 0x7f6830004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6847fff700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6830003a30 0x7f6830003e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f683800c1c0 tx=0x7f6830005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc35445da964ab6b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838007b50 con 0x7f6830003a30 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683800cc80 con 0x7f6830003a30 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.171+0000 7f6846ffd700 1 -- 172.21.15.79:0/3848120297 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380058a0 con 0x7f6830003a30 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 -- 172.21.15.79:0/3848120297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6830003a30 msgr2=0x7f6830003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6830003a30 0x7f6830003e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f683800c1c0 tx=0x7f6830005880 comp rx=0 tx=0).stop 2023-06-08T03:50:06.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 -- 172.21.15.79:0/3848120297 shutdown_connections 2023-06-08T03:50:06.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 --2- 172.21.15.79:0/3848120297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6830003a30 0x7f6830003e40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 -- 172.21.15.79:0/3848120297 >> 172.21.15.79:0/3848120297 conn(0x7f68480ced30 msgr2=0x7f68480cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:06.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 -- 172.21.15.79:0/3848120297 shutdown_connections 2023-06-08T03:50:06.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.172+0000 7f684e835700 1 -- 172.21.15.79:0/3848120297 wait complete. 2023-06-08T03:50:06.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.173+0000 7f684e835700 1 Processor -- start 2023-06-08T03:50:06.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.173+0000 7f684e835700 1 -- start start 2023-06-08T03:50:06.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684e835700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830003a30 0x7f68481df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684e835700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 0x7f68480c2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684e835700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68481f02c0 0x7f68481f0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684e835700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68481f0ca0 con 0x7f6848157bf0 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684e835700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68480c5e00 con 0x7f68481f02c0 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684e835700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68480c60e0 con 0x7f6830003a30 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f68477fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 0x7f68480c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f68477fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 0x7f68480c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33774/0 (socket says 172.21.15.79:33774) 2023-06-08T03:50:06.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f68477fe700 1 -- 172.21.15.79:0/2800058270 learned_addr learned my addr 172.21.15.79:0/2800058270 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f6847fff700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830003a30 0x7f68481df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.174+0000 7f684cdd2700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68481f02c0 0x7f68481f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f68477fe700 1 -- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830003a30 msgr2=0x7f68481df7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f68477fe700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830003a30 0x7f68481df7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f68477fe700 1 -- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68481f02c0 msgr2=0x7f68481f0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f68477fe700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68481f02c0 0x7f68481f0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f68477fe700 1 -- 172.21.15.79:0/2800058270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68481e0090 con 0x7f6848157bf0 2023-06-08T03:50:06.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f684cdd2700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68481f02c0 0x7f68481f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f68477fe700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 0x7f68480c2890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f683c012780 tx=0x7f683c012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c017070 con 0x7f6848157bf0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c0107f0 con 0x7f6848157bf0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68481e0370 con 0x7f6848157bf0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c010b60 con 0x7f6848157bf0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68481dfcf0 con 0x7f6848157bf0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f683c025440 con 0x7f6848157bf0 2023-06-08T03:50:06.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.175+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f683c01b460 con 0x7f6848157bf0 2023-06-08T03:50:06.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.176+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684806c2a0 con 0x7f6848157bf0 2023-06-08T03:50:06.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.178+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f683c02a060 con 0x7f6848157bf0 2023-06-08T03:50:06.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.250+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f68480cd8e0 con 0x7f6848157bf0 2023-06-08T03:50:06.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f6844ff9700 1 -- 172.21.15.79:0/2800058270 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f683c020070 con 0x7f6848157bf0 2023-06-08T03:50:06.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 msgr2=0x7f68480c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 0x7f68480c2890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f683c012780 tx=0x7f683c012a90 comp rx=0 tx=0).stop 2023-06-08T03:50:06.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 shutdown_connections 2023-06-08T03:50:06.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f68481f02c0 0x7f68481f0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6830003a30 0x7f68481df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 --2- 172.21.15.79:0/2800058270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6848157bf0 0x7f68480c2890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.253+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 >> 172.21.15.79:0/2800058270 conn(0x7f68480ced30 msgr2=0x7f68480ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:06.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.254+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 shutdown_connections 2023-06-08T03:50:06.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:06.254+0000 7f684e835700 1 -- 172.21.15.79:0/2800058270 wait complete. 2023-06-08T03:50:06.259 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2023-06-08T03:50:06.269 INFO:tasks.ceph:check_enable_crimson: False 2023-06-08T03:50:06.269 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-08T03:50:06.269 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:06.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-08T03:50:06.288 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:06.285+0000 7f7ea6d71200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-08T03:50:06.295 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:06.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-08T03:50:06.353 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:06.350+0000 7f7ea6d71200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-08T03:50:06.361 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:06.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-08T03:50:06.427 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:06.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-08T03:50:06.428 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:06.425+0000 7f7ea6d71200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-08T03:50:06.493 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:06.494 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-08T03:50:06.507 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:06.504+0000 7f7ea6d71200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-08T03:50:06.519 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:06.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-08T03:50:06.585 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:06.586 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-08T03:50:06.639 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:06.635+0000 7f90f4928200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-08T03:50:06.651 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:50:06.651 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-08T03:50:06.660 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:06.657+0000 7f7ea6d71200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-08T03:50:06.686 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 3ca6ef60-f85f-4f89-8729-e7e64956e1a1 0 2023-06-08T03:50:06.704 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:06.700+0000 7f90f4928200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-08T03:50:06.779 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:06.775+0000 7f90f4928200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-08T03:50:06.847 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.844+0000 7fe90c2d3700 1 Processor -- start 2023-06-08T03:50:06.848 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.844+0000 7fe90c2d3700 1 -- start start 2023-06-08T03:50:06.849 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90c2d3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 0x7fe90406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90c2d3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 0x7fe9040cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90c2d3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe9040cb1e0 0x7fe90406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90c2d3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe904069d50 con 0x7fe9040cb1e0 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90c2d3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040cd210 con 0x7fe9040cc900 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90c2d3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90415a460 con 0x7fe9040ae150 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90986e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 0x7fe9040cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90a06f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 0x7fe90406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.850 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90a06f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 0x7fe90406c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51286/0 (socket says 172.21.15.156:51286) 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90986e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 0x7fe9040cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47694/0 (socket says 172.21.15.156:47694) 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90a06f700 1 -- 172.21.15.156:0/445386418 learned_addr learned my addr 172.21.15.156:0/445386418 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.845+0000 7fe90a870700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.79:3300/0 conn(0x7fe9040cb1e0 0x7fe90406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90986e700 1 -- 172.21.15.156:0/445386418 >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 msgr2=0x7fe90406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90986e700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 0x7fe90406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90986e700 1 -- 172.21.15.156:0/445386418 >> v2:172.21.15.79:3300/0 conn(0x7fe9040cb1e0 msgr2=0x7fe90406aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:06.851 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90986e700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.79:3300/0 conn(0x7fe9040cb1e0 0x7fe90406aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90986e700 1 -- 172.21.15.156:0/445386418 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe904069f60 con 0x7fe9040ae150 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90a870700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.79:3300/0 conn(0x7fe9040cb1e0 0x7fe90406aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90a06f700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 0x7fe90406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90986e700 1 --2- 172.21.15.156:0/445386418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 0x7fe9040cccd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe904065b70 tx=0x7fe900009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39cd7774d9872c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90906d700 1 -- 172.21.15.156:0/445386418 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900011830 con 0x7fe9040ae150 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90906d700 1 -- 172.21.15.156:0/445386418 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe900011990 con 0x7fe9040ae150 2023-06-08T03:50:06.852 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.846+0000 7fe90906d700 1 -- 172.21.15.156:0/445386418 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900011d20 con 0x7fe9040ae150 2023-06-08T03:50:06.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 -- 172.21.15.156:0/445386418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 msgr2=0x7fe9040cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/445386418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 0x7fe9040cccd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe904065b70 tx=0x7fe900009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:06.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 -- 172.21.15.156:0/445386418 shutdown_connections 2023-06-08T03:50:06.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.79:3300/0 conn(0x7fe9040cb1e0 0x7fe90406aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/445386418 >> v2:172.21.15.156:3300/0 conn(0x7fe9040cc900 0x7fe90406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.855 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/445386418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040ae150 0x7fe9040cccd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 -- 172.21.15.156:0/445386418 >> 172.21.15.156:0/445386418 conn(0x7fe9040ced30 msgr2=0x7fe9040cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 -- 172.21.15.156:0/445386418 shutdown_connections 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.847+0000 7fe90c2d3700 1 -- 172.21.15.156:0/445386418 wait complete. 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.848+0000 7fe90c2d3700 1 Processor -- start 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.848+0000 7fe90c2d3700 1 -- start start 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9040ae150 0x7fe904158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.856 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 0x7fe9040c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9040cc900 0x7fe9040c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041e0b60 con 0x7fe9040cc900 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041dff90 con 0x7fe9040ae150 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9041e0230 con 0x7fe9040cb1e0 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90986e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 0x7fe9040c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90986e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 0x7fe9040c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47700/0 (socket says 172.21.15.156:47700) 2023-06-08T03:50:06.857 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9040ae150 0x7fe904158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90986e700 1 -- 172.21.15.156:0/2353975232 learned_addr learned my addr 172.21.15.156:0/2353975232 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a870700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9040cc900 0x7fe9040c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 -- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 msgr2=0x7fe9040c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 0x7fe9040c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 -- 172.21.15.156:0/2353975232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9040cc900 msgr2=0x7fe9040c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9040cc900 0x7fe9040c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.858 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 -- 172.21.15.156:0/2353975232 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f4009550 con 0x7fe9040ae150 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90986e700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 0x7fe9040c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90a06f700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9040ae150 0x7fe904158250 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e9a0 tx=0x7fe8f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004030 con 0x7fe9040ae150 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f4004190 con 0x7fe9040ae150 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004520 con 0x7fe9040ae150 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.849+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9000094a0 con 0x7fe9040ae150 2023-06-08T03:50:06.859 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.850+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9041e07f0 con 0x7fe9040ae150 2023-06-08T03:50:06.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.850+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe8f4005d20 con 0x7fe9040ae150 2023-06-08T03:50:06.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.850+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe8f40039a0 con 0x7fe9040ae150 2023-06-08T03:50:06.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.850+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9041e0980 con 0x7fe9040ae150 2023-06-08T03:50:06.860 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:06.853+0000 7f90f4928200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-08T03:50:06.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.853+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe8f4003c60 con 0x7fe9040ae150 2023-06-08T03:50:06.938 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.935+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3ca6ef60-f85f-4f89-8729-e7e64956e1a1", "id": 0} v 0) v1 -- 0x7fe90406f1d0 con 0x7fe9040ae150 2023-06-08T03:50:06.941 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.938+0000 7fe8faffd700 1 -- 172.21.15.156:0/2353975232 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ca6ef60-f85f-4f89-8729-e7e64956e1a1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe8f40118c0 con 0x7fe9040ae150 2023-06-08T03:50:06.942 INFO:teuthology.orchestra.run.smithi156.stdout:0 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9040ae150 msgr2=0x7fe904158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9040ae150 0x7fe904158250 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e9a0 tx=0x7fe8f400ecb0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 shutdown_connections 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9040cc900 0x7fe9040c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9040ae150 0x7fe904158250 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 --2- 172.21.15.156:0/2353975232 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe9040cb1e0 0x7fe9040c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:06.943 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 >> 172.21.15.156:0/2353975232 conn(0x7fe9040ced30 msgr2=0x7fe9040af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:06.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 shutdown_connections 2023-06-08T03:50:06.944 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:06.939+0000 7fe90c2d3700 1 -- 172.21.15.156:0/2353975232 wait complete. 2023-06-08T03:50:06.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new e5277170-8966-49c0-8c10-32be4c9d1814 1 2023-06-08T03:50:06.998 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:06.994+0000 7f90f4928200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-08T03:50:07.013 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:07.010+0000 7f7ea6d71200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-08T03:50:07.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.070+0000 7f17266c3700 1 Processor -- start 2023-06-08T03:50:07.073 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.070+0000 7f17266c3700 1 -- start start 2023-06-08T03:50:07.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f17266c3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 0x7f17200ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f17266c3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f172015d3d0 0x7f17200cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f17266c3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f17200cc4c0 0x7f172006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f17266c3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200cc920 con 0x7f172015d3d0 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f17266c3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200cd7b0 con 0x7f17200cc4c0 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f17266c3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200cda50 con 0x7f1720158d50 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 0x7f17200ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f1724c60700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f17200cc4c0 0x7f172006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f1724c60700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f17200cc4c0 0x7f172006eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51300/0 (socket says 172.21.15.156:51300) 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 0x7f17200ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47704/0 (socket says 172.21.15.156:47704) 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f1724c60700 1 -- 172.21.15.156:0/709817969 learned_addr learned my addr 172.21.15.156:0/709817969 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171f7fe700 1 --2- 172.21.15.156:0/709817969 >> v2:172.21.15.79:3300/0 conn(0x7f172015d3d0 0x7f17200cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 -- 172.21.15.156:0/709817969 >> v2:172.21.15.156:3300/0 conn(0x7f17200cc4c0 msgr2=0x7f172006eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 --2- 172.21.15.156:0/709817969 >> v2:172.21.15.156:3300/0 conn(0x7f17200cc4c0 0x7f172006eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 -- 172.21.15.156:0/709817969 >> v2:172.21.15.79:3300/0 conn(0x7f172015d3d0 msgr2=0x7f17200cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:07.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 --2- 172.21.15.156:0/709817969 >> v2:172.21.15.79:3300/0 conn(0x7f172015d3d0 0x7f17200cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 -- 172.21.15.156:0/709817969 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200cdd30 con 0x7f1720158d50 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171f7fe700 1 --2- 172.21.15.156:0/709817969 >> v2:172.21.15.79:3300/0 conn(0x7f172015d3d0 0x7f17200cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171ffff700 1 --2- 172.21.15.156:0/709817969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 0x7f17200ca360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1720070590 tx=0x7f1708009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811308e60d10946d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.071+0000 7f171effd700 1 -- 172.21.15.156:0/709817969 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708011830 con 0x7f1720158d50 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f171effd700 1 -- 172.21.15.156:0/709817969 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1708011990 con 0x7f1720158d50 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f171effd700 1 -- 172.21.15.156:0/709817969 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708011d20 con 0x7f1720158d50 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 -- 172.21.15.156:0/709817969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 msgr2=0x7f17200ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 --2- 172.21.15.156:0/709817969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 0x7f17200ca360 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1720070590 tx=0x7f1708009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 -- 172.21.15.156:0/709817969 shutdown_connections 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 --2- 172.21.15.156:0/709817969 >> v2:172.21.15.156:3300/0 conn(0x7f17200cc4c0 0x7f172006eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 --2- 172.21.15.156:0/709817969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1720158d50 0x7f17200ca360 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 --2- 172.21.15.156:0/709817969 >> v2:172.21.15.79:3300/0 conn(0x7f172015d3d0 0x7f17200cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 -- 172.21.15.156:0/709817969 >> 172.21.15.156:0/709817969 conn(0x7f17200b1d30 msgr2=0x7f172006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 -- 172.21.15.156:0/709817969 shutdown_connections 2023-06-08T03:50:07.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.072+0000 7f17266c3700 1 -- 172.21.15.156:0/709817969 wait complete. 2023-06-08T03:50:07.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.073+0000 7f17266c3700 1 Processor -- start 2023-06-08T03:50:07.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 -- start start 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 0x7f1720142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1720158d50 0x7f1720161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172015d3d0 0x7f17200b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1720162b60 con 0x7f1720158d50 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17201625a0 con 0x7f172015d3d0 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f17266c3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1720162880 con 0x7f17200cc4c0 2023-06-08T03:50:07.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f171ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 0x7f1720142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f171ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 0x7f1720142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47710/0 (socket says 172.21.15.156:47710) 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f1724c60700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172015d3d0 0x7f17200b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f171ffff700 1 -- 172.21.15.156:0/3457482801 learned_addr learned my addr 172.21.15.156:0/3457482801 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.074+0000 7f171f7fe700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1720158d50 0x7f1720161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171ffff700 1 -- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172015d3d0 msgr2=0x7f17200b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171ffff700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172015d3d0 0x7f17200b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.083 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171ffff700 1 -- 172.21.15.156:0/3457482801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1720158d50 msgr2=0x7f1720161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171ffff700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1720158d50 0x7f1720161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171ffff700 1 -- 172.21.15.156:0/3457482801 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080094a0 con 0x7f17200cc4c0 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f1724c60700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172015d3d0 0x7f17200b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171ffff700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 0x7f1720142fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1708000c00 tx=0x7f1708004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708004450 con 0x7f17200cc4c0 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170801e070 con 0x7f17200cc4c0 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17201427d0 con 0x7f17200cc4c0 2023-06-08T03:50:07.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080038e0 con 0x7f17200cc4c0 2023-06-08T03:50:07.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1720143510 con 0x7f17200cc4c0 2023-06-08T03:50:07.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1708003d40 con 0x7f17200cc4c0 2023-06-08T03:50:07.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.075+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f170801a7d0 con 0x7f17200cc4c0 2023-06-08T03:50:07.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.076+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1720142cf0 con 0x7f17200cc4c0 2023-06-08T03:50:07.085 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.078+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f1720142cf0 con 0x7f17200cc4c0 2023-06-08T03:50:07.092 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:07.089+0000 7f7ea6d71200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-08T03:50:07.167 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.163+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e5277170-8966-49c0-8c10-32be4c9d1814", "id": 1} v 0) v1 -- 0x7f1720073e90 con 0x7f17200cc4c0 2023-06-08T03:50:07.169 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.166+0000 7f171cff9700 1 -- 172.21.15.156:0/3457482801 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5277170-8966-49c0-8c10-32be4c9d1814", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1708020070 con 0x7f17200cc4c0 2023-06-08T03:50:07.170 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-06-08T03:50:07.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 msgr2=0x7f1720142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.171 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 0x7f1720142fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1708000c00 tx=0x7f1708004750 comp rx=0 tx=0).stop 2023-06-08T03:50:07.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 shutdown_connections 2023-06-08T03:50:07.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f172015d3d0 0x7f17200b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17200cc4c0 0x7f1720142fd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.172 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 --2- 172.21.15.156:0/3457482801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1720158d50 0x7f1720161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 >> 172.21.15.156:0/3457482801 conn(0x7f17200b1d30 msgr2=0x7f17200aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 shutdown_connections 2023-06-08T03:50:07.173 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.167+0000 7f17266c3700 1 -- 172.21.15.156:0/3457482801 wait complete. 2023-06-08T03:50:07.183 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new b3f37967-178c-4d15-9441-a4b57cd8b602 2 2023-06-08T03:50:07.237 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:07.234+0000 7f7ea6d71200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-08T03:50:07.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.300+0000 7f9a8ef9f700 1 Processor -- start 2023-06-08T03:50:07.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.300+0000 7f9a8ef9f700 1 -- start start 2023-06-08T03:50:07.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8ef9f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9a880d0240 0x7f9a880af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8ef9f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8815d930 con 0x7f9a880cd000 2023-06-08T03:50:07.305 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8ef9f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8815e770 con 0x7f9a880ce3f0 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8ef9f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8815db60 con 0x7f9a880d0240 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8d53c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9a880ce3f0 0x7f9a880ce870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43374/0 (socket says 172.21.15.156:43374) 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8d53c700 1 -- 172.21.15.156:0/1021112783 learned_addr learned my addr 172.21.15.156:0/1021112783 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a8cd3b700 1 --2- 172.21.15.156:0/1021112783 >> v2:172.21.15.79:3300/0 conn(0x7f9a880d0240 0x7f9a880af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248592331 0 0) 0x7f9a8815e770 con 0x7f9a880ce3f0 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a780035b0 con 0x7f9a880ce3f0 2023-06-08T03:50:07.306 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476259065 0 0) 0x7f9a8815d930 con 0x7f9a880cd000 2023-06-08T03:50:07.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.301+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a8815e770 con 0x7f9a880cd000 2023-06-08T03:50:07.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8cd3b700 1 -- 172.21.15.156:0/1021112783 >> v1:172.21.15.156:6789/0 conn(0x7f9a880ce3f0 legacy=0x7f9a880ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8cd3b700 1 -- 172.21.15.156:0/1021112783 >> v1:172.21.15.79:6789/0 conn(0x7f9a880cd000 legacy=0x7f9a880cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8cd3b700 1 -- 172.21.15.156:0/1021112783 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8815de40 con 0x7f9a880d0240 2023-06-08T03:50:07.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8cd3b700 1 --2- 172.21.15.156:0/1021112783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880d0240 0x7f9a880af7d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009840 tx=0x7f9a7c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539d0d0d0914c91f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.307 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c011830 con 0x7f9a880d0240 2023-06-08T03:50:07.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c011990 con 0x7f9a880d0240 2023-06-08T03:50:07.308 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a877fe700 1 -- 172.21.15.156:0/1021112783 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c011d20 con 0x7f9a880d0240 2023-06-08T03:50:07.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1021112783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880d0240 msgr2=0x7f9a880af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8ef9f700 1 --2- 172.21.15.156:0/1021112783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880d0240 0x7f9a880af7d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009840 tx=0x7f9a7c009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:07.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1021112783 shutdown_connections 2023-06-08T03:50:07.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8ef9f700 1 --2- 172.21.15.156:0/1021112783 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880d0240 0x7f9a880af7d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.302+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1021112783 >> 172.21.15.156:0/1021112783 conn(0x7f9a8806e1e0 msgr2=0x7f9a8806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.303+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1021112783 shutdown_connections 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.303+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1021112783 wait complete. 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.304+0000 7f9a8ef9f700 1 Processor -- start 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.304+0000 7f9a8ef9f700 1 -- start start 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880cd000 0x7f9a8814e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 0x7f9a8806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.311 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a880d0240 0x7f9a8806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c85f0 con 0x7f9a880cd000 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c7740 con 0x7f9a880ce3f0 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c79e0 con 0x7f9a880d0240 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a87fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 0x7f9a8806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a880d0240 0x7f9a8806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a87fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 0x7f9a8806c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51316/0 (socket says 172.21.15.156:51316) 2023-06-08T03:50:07.312 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a87fff700 1 -- 172.21.15.156:0/1341580756 learned_addr learned my addr 172.21.15.156:0/1341580756 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8cd3b700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880cd000 0x7f9a8814e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 -- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 msgr2=0x7f9a8806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 0x7f9a8806c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 -- 172.21.15.156:0/1341580756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880cd000 msgr2=0x7f9a8814e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880cd000 0x7f9a8814e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 -- 172.21.15.156:0/1341580756 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0094a0 con 0x7f9a880d0240 2023-06-08T03:50:07.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a87fff700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 0x7f9a8806c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8d53c700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a880d0240 0x7f9a8806c910 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000ca70 tx=0x7f9a8000cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000a9e0 con 0x7f9a880d0240 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8000ab40 con 0x7f9a880d0240 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880c7cc0 con 0x7f9a880d0240 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80012400 con 0x7f9a880d0240 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.305+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a880c8280 con 0x7f9a880d0240 2023-06-08T03:50:07.314 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.306+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9a80012560 con 0x7f9a880d0240 2023-06-08T03:50:07.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.306+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f9a8001b7d0 con 0x7f9a880d0240 2023-06-08T03:50:07.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.307+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a880c8760 con 0x7f9a880d0240 2023-06-08T03:50:07.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.309+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9a80017070 con 0x7f9a880d0240 2023-06-08T03:50:07.323 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:07.320+0000 7f7ea6d71200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-08T03:50:07.335 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:07.331+0000 7f90f4928200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-08T03:50:07.394 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.391+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b3f37967-178c-4d15-9441-a4b57cd8b602", "id": 2} v 0) v1 -- 0x7f9a88073e90 con 0x7f9a880d0240 2023-06-08T03:50:07.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.399+0000 7f9a857fa700 1 -- 172.21.15.156:0/1341580756 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3f37967-178c-4d15-9441-a4b57cd8b602", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9a8001b480 con 0x7f9a880d0240 2023-06-08T03:50:07.402 INFO:teuthology.orchestra.run.smithi156.stdout:2 2023-06-08T03:50:07.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a880d0240 msgr2=0x7f9a8806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a880d0240 0x7f9a8806c910 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000ca70 tx=0x7f9a8000cd80 comp rx=0 tx=0).stop 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 shutdown_connections 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a880d0240 0x7f9a8806c910 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a880cd000 0x7f9a8814e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 --2- 172.21.15.156:0/1341580756 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a880ce3f0 0x7f9a8806c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 >> 172.21.15.156:0/1341580756 conn(0x7f9a8806e1e0 msgr2=0x7f9a880b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 shutdown_connections 2023-06-08T03:50:07.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.400+0000 7f9a8ef9f700 1 -- 172.21.15.156:0/1341580756 wait complete. 2023-06-08T03:50:07.410 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:07.406+0000 7f90f4928200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-08T03:50:07.416 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new a6ab1aab-9727-482c-8b65-faf5925615a6 3 2023-06-08T03:50:07.539 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.536+0000 7f98dbf0a700 1 Processor -- start 2023-06-08T03:50:07.540 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.536+0000 7f98dbf0a700 1 -- start start 2023-06-08T03:50:07.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98dbf0a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f98d415fb10 0x7f98d4070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98dbf0a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d40cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.541 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98dbf0a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d4070bd0 con 0x7f98d40cce80 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98dbf0a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d40cad50 con 0x7f98d415fb10 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98dbf0a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d40ce570 con 0x7f98d40cb1e0 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98d9ca6700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f98d40cce80 0x7f98d40cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43382/0 (socket says 172.21.15.156:43382) 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98da4a7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d40cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98d9ca6700 1 -- 172.21.15.156:0/1800049147 learned_addr learned my addr 172.21.15.156:0/1800049147 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.537+0000 7f98d94a5700 1 --2- 172.21.15.156:0/1800049147 >> v2:172.21.15.79:3300/0 conn(0x7f98d415fb10 0x7f98d4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.542 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98da4a7700 1 -- 172.21.15.156:0/1800049147 >> v2:172.21.15.79:3300/0 conn(0x7f98d415fb10 msgr2=0x7f98d4070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98d8ca4700 1 -- 172.21.15.156:0/1800049147 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617286249 0 0) 0x7f98c4001110 con 0x7f98d40cce80 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98da4a7700 1 --2- 172.21.15.156:0/1800049147 >> v2:172.21.15.79:3300/0 conn(0x7f98d415fb10 0x7f98d4070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98da4a7700 1 -- 172.21.15.156:0/1800049147 >> v1:172.21.15.156:6789/0 conn(0x7f98d40cce80 legacy=0x7f98d40cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98da4a7700 1 -- 172.21.15.156:0/1800049147 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4160080 con 0x7f98d40cb1e0 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98d94a5700 1 --2- 172.21.15.156:0/1800049147 >> v2:172.21.15.79:3300/0 conn(0x7f98d415fb10 0x7f98d4070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98da4a7700 1 --2- 172.21.15.156:0/1800049147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d40cb680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009d90 tx=0x7f98cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9d3b8aa2eb5bea7 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98d8ca4700 1 -- 172.21.15.156:0/1800049147 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00e070 con 0x7f98d40cb1e0 2023-06-08T03:50:07.543 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98d8ca4700 1 -- 172.21.15.156:0/1800049147 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc00bbd0 con 0x7f98d40cb1e0 2023-06-08T03:50:07.544 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98d8ca4700 1 -- 172.21.15.156:0/1800049147 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc012430 con 0x7f98d40cb1e0 2023-06-08T03:50:07.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98dbf0a700 1 -- 172.21.15.156:0/1800049147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 msgr2=0x7f98d40cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.538+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/1800049147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d40cb680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009d90 tx=0x7f98cc00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:07.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.539+0000 7f98dbf0a700 1 -- 172.21.15.156:0/1800049147 shutdown_connections 2023-06-08T03:50:07.546 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.539+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/1800049147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d40cb680 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.539+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/1800049147 >> v2:172.21.15.79:3300/0 conn(0x7f98d415fb10 0x7f98d4070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.539+0000 7f98dbf0a700 1 -- 172.21.15.156:0/1800049147 >> 172.21.15.156:0/1800049147 conn(0x7f98d406e1e0 msgr2=0x7f98d406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.539+0000 7f98dbf0a700 1 -- 172.21.15.156:0/1800049147 shutdown_connections 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.539+0000 7f98dbf0a700 1 -- 172.21.15.156:0/1800049147 wait complete. 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.540+0000 7f98dbf0a700 1 Processor -- start 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.540+0000 7f98dbf0a700 1 -- start start 2023-06-08T03:50:07.547 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.540+0000 7f98dbf0a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.540+0000 7f98dbf0a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98d40cce80 0x7f98d4153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98dbf0a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d415fb10 0x7f98d415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98dbf0a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d40b0a00 con 0x7f98d40cce80 2023-06-08T03:50:07.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98dbf0a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d415c770 con 0x7f98d415fb10 2023-06-08T03:50:07.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98dbf0a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d41540f0 con 0x7f98d40cb1e0 2023-06-08T03:50:07.548 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d415c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47734/0 (socket says 172.21.15.156:47734) 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 -- 172.21.15.156:0/851675852 learned_addr learned my addr 172.21.15.156:0/851675852 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98da4a7700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d415fb10 0x7f98d415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d94a5700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98d40cce80 0x7f98d4153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 -- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d415fb10 msgr2=0x7f98d415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d415fb10 0x7f98d415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 -- 172.21.15.156:0/851675852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98d40cce80 msgr2=0x7f98d4153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.549 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98d40cce80 0x7f98d4153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 -- 172.21.15.156:0/851675852 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009a70 con 0x7f98d40cb1e0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98d9ca6700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d415c230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c580 tx=0x7f98c400c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4004c80 con 0x7f98d40cb1e0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c4004e00 con 0x7f98d40cb1e0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4007310 con 0x7f98d40cb1e0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d415a580 con 0x7f98d40cb1e0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d40b00d0 con 0x7f98d40cb1e0 2023-06-08T03:50:07.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f98c401d3f0 con 0x7f98d40cb1e0 2023-06-08T03:50:07.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.541+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f98c4013850 con 0x7f98d40cb1e0 2023-06-08T03:50:07.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.542+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d41608d0 con 0x7f98d40cb1e0 2023-06-08T03:50:07.551 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:07.542+0000 7f90f4928200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-08T03:50:07.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.544+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f98d4070bd0 con 0x7f98d40cb1e0 2023-06-08T03:50:07.626 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:07.622+0000 7f90f4928200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-08T03:50:07.631 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.627+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a6ab1aab-9727-482c-8b65-faf5925615a6", "id": 3} v 0) v1 -- 0x7f98d4073e90 con 0x7f98d40cb1e0 2023-06-08T03:50:07.636 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.633+0000 7f98ca7fc700 1 -- 172.21.15.156:0/851675852 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a6ab1aab-9727-482c-8b65-faf5925615a6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98c4013570 con 0x7f98d40cb1e0 2023-06-08T03:50:07.637 INFO:teuthology.orchestra.run.smithi156.stdout:3 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 msgr2=0x7f98d415c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d415c230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c580 tx=0x7f98c400c890 comp rx=0 tx=0).stop 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 shutdown_connections 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d415fb10 0x7f98d415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d40cb1e0 0x7f98d415c230 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 --2- 172.21.15.156:0/851675852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98d40cce80 0x7f98d4153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 >> 172.21.15.156:0/851675852 conn(0x7f98d406e1e0 msgr2=0x7f98d406ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 shutdown_connections 2023-06-08T03:50:07.639 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.634+0000 7f98dbf0a700 1 -- 172.21.15.156:0/851675852 wait complete. 2023-06-08T03:50:07.649 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new b6d3f147-10af-4fb9-94ee-49b2b58dc603 4 2023-06-08T03:50:07.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.767+0000 7fe15e5dc700 1 Processor -- start 2023-06-08T03:50:07.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.767+0000 7fe15e5dc700 1 -- start start 2023-06-08T03:50:07.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15e5dc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1580d0060 0x7fe1580af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15e5dc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe1580cd080 0x7fe1580cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15e5dc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580cae50 con 0x7fe1580b20f0 2023-06-08T03:50:07.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15e5dc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580cbea0 con 0x7fe1580cd080 2023-06-08T03:50:07.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15e5dc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580cda30 con 0x7fe1580d0060 2023-06-08T03:50:07.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe157fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1580d0060 0x7fe1580af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15cb79700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fe1580b20f0 0x7fe1580b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:43394/0 (socket says 172.21.15.156:43394) 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe15cb79700 1 -- 172.21.15.156:0/739352638 learned_addr learned my addr 172.21.15.156:0/739352638 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.768+0000 7fe1577fe700 1 --2- 172.21.15.156:0/739352638 >> v2:172.21.15.79:3300/0 conn(0x7fe1580cd080 0x7fe1580cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe157fff700 1 -- 172.21.15.156:0/739352638 >> v2:172.21.15.79:3300/0 conn(0x7fe1580cd080 msgr2=0x7fe1580cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe157fff700 1 --2- 172.21.15.156:0/739352638 >> v2:172.21.15.79:3300/0 conn(0x7fe1580cd080 0x7fe1580cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe157fff700 1 -- 172.21.15.156:0/739352638 >> v1:172.21.15.156:6789/0 conn(0x7fe1580b20f0 legacy=0x7fe1580b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe157fff700 1 -- 172.21.15.156:0/739352638 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15815d930 con 0x7fe1580d0060 2023-06-08T03:50:07.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe157fff700 1 --2- 172.21.15.156:0/739352638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580d0060 0x7fe1580af7d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe15815d890 tx=0x7fe148009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22e1fcfe39c9c280 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe156ffd700 1 -- 172.21.15.156:0/739352638 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148011830 con 0x7fe1580d0060 2023-06-08T03:50:07.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe156ffd700 1 -- 172.21.15.156:0/739352638 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148011990 con 0x7fe1580d0060 2023-06-08T03:50:07.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe156ffd700 1 -- 172.21.15.156:0/739352638 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148011d20 con 0x7fe1580d0060 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe15e5dc700 1 -- 172.21.15.156:0/739352638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580d0060 msgr2=0x7fe1580af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.769+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/739352638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580d0060 0x7fe1580af7d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe15815d890 tx=0x7fe148009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.770+0000 7fe15e5dc700 1 -- 172.21.15.156:0/739352638 shutdown_connections 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.770+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/739352638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580d0060 0x7fe1580af7d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.770+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/739352638 >> v2:172.21.15.79:3300/0 conn(0x7fe1580cd080 0x7fe1580cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.770+0000 7fe15e5dc700 1 -- 172.21.15.156:0/739352638 >> 172.21.15.156:0/739352638 conn(0x7fe15806e1e0 msgr2=0x7fe15806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.770+0000 7fe15e5dc700 1 -- 172.21.15.156:0/739352638 shutdown_connections 2023-06-08T03:50:07.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.770+0000 7fe15e5dc700 1 -- 172.21.15.156:0/739352638 wait complete. 2023-06-08T03:50:07.778 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.771+0000 7fe15e5dc700 1 Processor -- start 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.771+0000 7fe15e5dc700 1 -- start start 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15e5dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580b20f0 0x7fe1581559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15e5dc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1580cd080 0x7fe1581e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15e5dc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1580d0060 0x7fe1581e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15e5dc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe158155ee0 con 0x7fe1580d0060 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15e5dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581e4140 con 0x7fe1580b20f0 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15e5dc700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581e4420 con 0x7fe1580cd080 2023-06-08T03:50:07.779 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe1577fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1580cd080 0x7fe1581e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe1577fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1580cd080 0x7fe1581e3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47748/0 (socket says 172.21.15.156:47748) 2023-06-08T03:50:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe1577fe700 1 -- 172.21.15.156:0/920988265 learned_addr learned my addr 172.21.15.156:0/920988265 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe157fff700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580b20f0 0x7fe1581559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.772+0000 7fe15cb79700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1580d0060 0x7fe1581e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe157fff700 1 -- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1580cd080 msgr2=0x7fe1581e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.780 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe157fff700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1580cd080 0x7fe1581e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe157fff700 1 -- 172.21.15.156:0/920988265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1580d0060 msgr2=0x7fe1581e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe157fff700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1580d0060 0x7fe1581e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe157fff700 1 -- 172.21.15.156:0/920988265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480094a0 con 0x7fe1580b20f0 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe157fff700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580b20f0 0x7fe1581559a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe148016000 tx=0x7fe148004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480036a0 con 0x7fe1580b20f0 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14801f070 con 0x7fe1580b20f0 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1581e4700 con 0x7fe1580b20f0 2023-06-08T03:50:07.781 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148003a40 con 0x7fe1580b20f0 2023-06-08T03:50:07.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.773+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1581e4cc0 con 0x7fe1580b20f0 2023-06-08T03:50:07.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.774+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe148026480 con 0x7fe1580b20f0 2023-06-08T03:50:07.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.774+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe158073e90 con 0x7fe1580b20f0 2023-06-08T03:50:07.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.774+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fe148021070 con 0x7fe1580b20f0 2023-06-08T03:50:07.782 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.778+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe148015020 con 0x7fe1580b20f0 2023-06-08T03:50:07.806 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:07.803+0000 7f7ea6d71200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-08T03:50:07.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.861+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b6d3f147-10af-4fb9-94ee-49b2b58dc603", "id": 4} v 0) v1 -- 0x7fe1580ca650 con 0x7fe1580b20f0 2023-06-08T03:50:07.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.867+0000 7fe154ff9700 1 -- 172.21.15.156:0/920988265 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6d3f147-10af-4fb9-94ee-49b2b58dc603", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe148026820 con 0x7fe1580b20f0 2023-06-08T03:50:07.870 INFO:teuthology.orchestra.run.smithi156.stdout:4 2023-06-08T03:50:07.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580b20f0 msgr2=0x7fe1581559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:07.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580b20f0 0x7fe1581559a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe148016000 tx=0x7fe148004750 comp rx=0 tx=0).stop 2023-06-08T03:50:07.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 shutdown_connections 2023-06-08T03:50:07.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe1580d0060 0x7fe1581e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1580b20f0 0x7fe1581559a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 --2- 172.21.15.156:0/920988265 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1580cd080 0x7fe1581e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:07.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 >> 172.21.15.156:0/920988265 conn(0x7fe15806e1e0 msgr2=0x7fe1580b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:07.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 shutdown_connections 2023-06-08T03:50:07.873 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:07.868+0000 7fe15e5dc700 1 -- 172.21.15.156:0/920988265 wait complete. 2023-06-08T03:50:07.884 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 8b340c4a-a17d-428b-8182-fa1a28f3d274 5 2023-06-08T03:50:08.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.002+0000 7fa63bb78700 1 Processor -- start 2023-06-08T03:50:08.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63bb78700 1 -- start start 2023-06-08T03:50:08.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63bb78700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa63406d410 0x7fa63406c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63bb78700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63406b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63bb78700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63415caa0 con 0x7fa63406a860 2023-06-08T03:50:08.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63bb78700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63415cd80 con 0x7fa63406d410 2023-06-08T03:50:08.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63bb78700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63415d060 con 0x7fa63406ae60 2023-06-08T03:50:08.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.003+0000 7fa63a115700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63406b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63406b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47756/0 (socket says 172.21.15.156:47756) 2023-06-08T03:50:08.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 -- 172.21.15.156:0/3321342638 learned_addr learned my addr 172.21.15.156:0/3321342638 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:08.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa639914700 1 --2- 172.21.15.156:0/3321342638 >> v2:172.21.15.79:3300/0 conn(0x7fa63406d410 0x7fa63406c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 -- 172.21.15.156:0/3321342638 >> v2:172.21.15.79:3300/0 conn(0x7fa63406d410 msgr2=0x7fa63406c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 --2- 172.21.15.156:0/3321342638 >> v2:172.21.15.79:3300/0 conn(0x7fa63406d410 0x7fa63406c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 -- 172.21.15.156:0/3321342638 >> v1:172.21.15.79:6789/0 conn(0x7fa63406a860 legacy=0x7fa63406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 -- 172.21.15.156:0/3321342638 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63415d340 con 0x7fa63406ae60 2023-06-08T03:50:08.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa638912700 1 -- 172.21.15.156:0/3321342638 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342825969 0 0) 0x7fa63415caa0 con 0x7fa63406a860 2023-06-08T03:50:08.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa63a115700 1 --2- 172.21.15.156:0/3321342638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63406b280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009e50 tx=0x7fa62c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58d5db691d4e5cb server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa638912700 1 -- 172.21.15.156:0/3321342638 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00e070 con 0x7fa63406ae60 2023-06-08T03:50:08.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa638912700 1 -- 172.21.15.156:0/3321342638 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c00b910 con 0x7fa63406ae60 2023-06-08T03:50:08.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.004+0000 7fa638912700 1 -- 172.21.15.156:0/3321342638 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00bcc0 con 0x7fa63406ae60 2023-06-08T03:50:08.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 -- 172.21.15.156:0/3321342638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 msgr2=0x7fa63406b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 --2- 172.21.15.156:0/3321342638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63406b280 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009e50 tx=0x7fa62c00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:08.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 -- 172.21.15.156:0/3321342638 shutdown_connections 2023-06-08T03:50:08.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 --2- 172.21.15.156:0/3321342638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63406b280 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 --2- 172.21.15.156:0/3321342638 >> v2:172.21.15.79:3300/0 conn(0x7fa63406d410 0x7fa63406c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 -- 172.21.15.156:0/3321342638 >> 172.21.15.156:0/3321342638 conn(0x7fa6340ced30 msgr2=0x7fa6340cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 -- 172.21.15.156:0/3321342638 shutdown_connections 2023-06-08T03:50:08.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.005+0000 7fa63bb78700 1 -- 172.21.15.156:0/3321342638 wait complete. 2023-06-08T03:50:08.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.006+0000 7fa63bb78700 1 Processor -- start 2023-06-08T03:50:08.016 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.006+0000 7fa63bb78700 1 -- start start 2023-06-08T03:50:08.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63406a860 0x7fa6340b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63409f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa63406d410 0x7fa6340af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6340b4ad0 con 0x7fa63406a860 2023-06-08T03:50:08.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63409cef0 con 0x7fa63406d410 2023-06-08T03:50:08.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63409d1d0 con 0x7fa63406ae60 2023-06-08T03:50:08.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63409f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63409f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47760/0 (socket says 172.21.15.156:47760) 2023-06-08T03:50:08.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63a115700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa63406d410 0x7fa6340af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 -- 172.21.15.156:0/2230865604 learned_addr learned my addr 172.21.15.156:0/2230865604 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:08.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639914700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63406a860 0x7fa6340b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 -- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa63406d410 msgr2=0x7fa6340af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa63406d410 0x7fa6340af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 -- 172.21.15.156:0/2230865604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63406a860 msgr2=0x7fa6340b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63406a860 0x7fa6340b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.023 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 -- 172.21.15.156:0/2230865604 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c009b30 con 0x7fa63406ae60 2023-06-08T03:50:08.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa639113700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63409f3f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6240011e0 tx=0x7fa62400ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240054e0 con 0x7fa63406ae60 2023-06-08T03:50:08.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624003680 con 0x7fa63406ae60 2023-06-08T03:50:08.024 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624003a30 con 0x7fa63406ae60 2023-06-08T03:50:08.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63409d4b0 con 0x7fa63406ae60 2023-06-08T03:50:08.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.007+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63409da70 con 0x7fa63406ae60 2023-06-08T03:50:08.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.008+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa62401c440 con 0x7fa63406ae60 2023-06-08T03:50:08.025 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.008+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa624012980 con 0x7fa63406ae60 2023-06-08T03:50:08.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.008+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63415ff60 con 0x7fa63406ae60 2023-06-08T03:50:08.026 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.010+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa63415ff60 con 0x7fa63406ae60 2023-06-08T03:50:08.063 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:08.059+0000 7f90f4928200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-08T03:50:08.114 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.111+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8b340c4a-a17d-428b-8182-fa1a28f3d274", "id": 5} v 0) v1 -- 0x7fa63409dc90 con 0x7fa63406ae60 2023-06-08T03:50:08.122 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.118+0000 7fa62a7fc700 1 -- 172.21.15.156:0/2230865604 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8b340c4a-a17d-428b-8182-fa1a28f3d274", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa624017070 con 0x7fa63406ae60 2023-06-08T03:50:08.122 INFO:teuthology.orchestra.run.smithi156.stdout:5 2023-06-08T03:50:08.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 msgr2=0x7fa63409f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63409f3f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6240011e0 tx=0x7fa62400ca30 comp rx=0 tx=0).stop 2023-06-08T03:50:08.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 shutdown_connections 2023-06-08T03:50:08.123 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa63406d410 0x7fa6340af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63406a860 0x7fa6340b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 --2- 172.21.15.156:0/2230865604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa63406ae60 0x7fa63409f3f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 >> 172.21.15.156:0/2230865604 conn(0x7fa6340ced30 msgr2=0x7fa6340cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 shutdown_connections 2023-06-08T03:50:08.124 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.119+0000 7fa63bb78700 1 -- 172.21.15.156:0/2230865604 wait complete. 2023-06-08T03:50:08.136 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 318c708a-4f08-4506-9c31-7f0b3f59b582 6 2023-06-08T03:50:08.286 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.282+0000 7faffa0d5700 1 Processor -- start 2023-06-08T03:50:08.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.283+0000 7faffa0d5700 1 -- start start 2023-06-08T03:50:08.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faffa0d5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40ca160 0x7faff40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faffa0d5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faff40af8d0 0x7faff40713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faffa0d5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faff4071920 0x7faff40c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faffa0d5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff40cda10 con 0x7faff40af8d0 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faffa0d5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff4071d80 con 0x7faff4071920 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faffa0d5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff40ca530 con 0x7faff40ca160 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faff3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faff4071920 0x7faff40c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faff3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faff4071920 0x7faff40c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51360/0 (socket says 172.21.15.156:51360) 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.284+0000 7faff3fff700 1 -- 172.21.15.156:0/2275214739 learned_addr learned my addr 172.21.15.156:0/2275214739 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff3fff700 1 -- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40ca160 msgr2=0x7faff40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff3fff700 1 --2- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40ca160 0x7faff40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff3fff700 1 -- 172.21.15.156:0/2275214739 >> v2:172.21.15.79:3300/0 conn(0x7faff40af8d0 msgr2=0x7faff40713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff3fff700 1 --2- 172.21.15.156:0/2275214739 >> v2:172.21.15.79:3300/0 conn(0x7faff40af8d0 0x7faff40713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff3fff700 1 -- 172.21.15.156:0/2275214739 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff40cb640 con 0x7faff4071920 2023-06-08T03:50:08.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff3fff700 1 --2- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c8b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faff40cf5a0 tx=0x7fafec009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da00ef3c7eb48d18 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff27fc700 1 -- 172.21.15.156:0/2275214739 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00b870 con 0x7faff4071920 2023-06-08T03:50:08.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff27fc700 1 -- 172.21.15.156:0/2275214739 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec00b9d0 con 0x7faff4071920 2023-06-08T03:50:08.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.285+0000 7faff27fc700 1 -- 172.21.15.156:0/2275214739 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00bd60 con 0x7faff4071920 2023-06-08T03:50:08.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 -- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 msgr2=0x7faff40c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 --2- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c8b30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faff40cf5a0 tx=0x7fafec009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 -- 172.21.15.156:0/2275214739 shutdown_connections 2023-06-08T03:50:08.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 --2- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c8b30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 --2- 172.21.15.156:0/2275214739 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40ca160 0x7faff40cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 --2- 172.21.15.156:0/2275214739 >> v2:172.21.15.79:3300/0 conn(0x7faff40af8d0 0x7faff40713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.292 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 -- 172.21.15.156:0/2275214739 >> 172.21.15.156:0/2275214739 conn(0x7faff40ce4e0 msgr2=0x7faff40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 -- 172.21.15.156:0/2275214739 shutdown_connections 2023-06-08T03:50:08.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.286+0000 7faffa0d5700 1 -- 172.21.15.156:0/2275214739 wait complete. 2023-06-08T03:50:08.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.288+0000 7faffa0d5700 1 Processor -- start 2023-06-08T03:50:08.293 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.288+0000 7faffa0d5700 1 -- start start 2023-06-08T03:50:08.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faffa0d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.294 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faffa0d5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40af8d0 0x7faff40967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faffa0d5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 0x7faff40c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faffa0d5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff40aeb30 con 0x7faff40ca160 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faffa0d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff4097a60 con 0x7faff4071920 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faffa0d5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff4097ef0 con 0x7faff40af8d0 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faff3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 0x7faff40c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faff3fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 0x7faff40c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.156:46938/0 (socket says 172.21.15.156:46938) 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faff3fff700 1 -- 172.21.15.156:0/1268955829 learned_addr learned my addr 172.21.15.156:0/1268955829 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:08.295 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.289+0000 7faff37fe700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faff3fff700 1 -- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40af8d0 msgr2=0x7faff40967e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faff3fff700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40af8d0 0x7faff40967e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faff3fff700 1 -- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 msgr2=0x7faff40c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faff3fff700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faff3fff700 1 -- 172.21.15.156:0/1268955829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec009550 con 0x7faff40ca160 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faff3fff700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 0x7faff40c6280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafec014040 tx=0x7fafec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec004310 con 0x7faff40ca160 2023-06-08T03:50:08.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff4097450 con 0x7faff40ca160 2023-06-08T03:50:08.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec01e070 con 0x7faff40ca160 2023-06-08T03:50:08.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0036a0 con 0x7faff40ca160 2023-06-08T03:50:08.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff40ae570 con 0x7faff40ca160 2023-06-08T03:50:08.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fafec020030 con 0x7faff40ca160 2023-06-08T03:50:08.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.290+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fafec023430 con 0x7faff40ca160 2023-06-08T03:50:08.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.291+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd80052f0 con 0x7faff40ca160 2023-06-08T03:50:08.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.294+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fafec015020 con 0x7faff40ca160 2023-06-08T03:50:08.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.400+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd new", "uuid": "318c708a-4f08-4506-9c31-7f0b3f59b582", "id": 6} v 0) v1 -- 0x7fafd8005910 con 0x7faff40ca160 2023-06-08T03:50:08.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.408+0000 7fafebfff700 1 -- 172.21.15.156:0/1268955829 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "318c708a-4f08-4506-9c31-7f0b3f59b582", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fafec02c440 con 0x7faff40ca160 2023-06-08T03:50:08.411 INFO:teuthology.orchestra.run.smithi156.stdout:6 2023-06-08T03:50:08.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 msgr2=0x7faff40c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 0x7faff40c6280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafec014040 tx=0x7fafec004750 comp rx=0 tx=0).stop 2023-06-08T03:50:08.412 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 shutdown_connections 2023-06-08T03:50:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faff40ca160 0x7faff40c6280 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faff4071920 0x7faff40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 --2- 172.21.15.156:0/1268955829 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faff40af8d0 0x7faff40967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 >> 172.21.15.156:0/1268955829 conn(0x7faff40ce4e0 msgr2=0x7faff415b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 shutdown_connections 2023-06-08T03:50:08.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.409+0000 7faffa0d5700 1 -- 172.21.15.156:0/1268955829 wait complete. 2023-06-08T03:50:08.425 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 735d6453-9a2d-45cd-9877-3ad4133082d6 7 2023-06-08T03:50:08.452 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:08.449+0000 7f7ea6d71200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-08T03:50:08.550 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.547+0000 7f4e1696e700 1 Processor -- start 2023-06-08T03:50:08.551 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.547+0000 7f4e1696e700 1 -- start start 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.547+0000 7f4e1696e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e10158ec0 0x7f4e1015d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e1696e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4e10071070 0x7f4e100cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e1696e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1015d8d0 con 0x7f4e100cc480 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e1696e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e100cc920 con 0x7f4e10071070 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e1696e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1006df90 con 0x7f4e10158ec0 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e10158ec0 0x7f4e1015d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.553 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4e10158ec0 0x7f4e1015d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51366/0 (socket says 172.21.15.156:51366) 2023-06-08T03:50:08.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 -- 172.21.15.156:0/4123193324 learned_addr learned my addr 172.21.15.156:0/4123193324 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:08.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0f7fe700 1 --2- 172.21.15.156:0/4123193324 >> v2:172.21.15.79:3300/0 conn(0x7f4e10071070 0x7f4e100cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 -- 172.21.15.156:0/4123193324 >> v2:172.21.15.79:3300/0 conn(0x7f4e10071070 msgr2=0x7f4e100cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 --2- 172.21.15.156:0/4123193324 >> v2:172.21.15.79:3300/0 conn(0x7f4e10071070 0x7f4e100cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 -- 172.21.15.156:0/4123193324 >> v1:172.21.15.79:6789/0 conn(0x7f4e100cc480 legacy=0x7f4e1006eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.554 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 -- 172.21.15.156:0/4123193324 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e100ca320 con 0x7f4e10158ec0 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0f7fe700 1 --2- 172.21.15.156:0/4123193324 >> v2:172.21.15.79:3300/0 conn(0x7f4e10071070 0x7f4e100cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0ffff700 1 --2- 172.21.15.156:0/4123193324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e1015d390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e100c2c10 tx=0x7f4e0000be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=261fda9bd2136dda server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0effd700 1 -- 172.21.15.156:0/4123193324 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000dcf0 con 0x7f4e10158ec0 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0effd700 1 -- 172.21.15.156:0/4123193324 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0000de50 con 0x7f4e10158ec0 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.548+0000 7f4e0effd700 1 -- 172.21.15.156:0/4123193324 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004250 con 0x7f4e10158ec0 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 -- 172.21.15.156:0/4123193324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 msgr2=0x7f4e1015d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.555 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 --2- 172.21.15.156:0/4123193324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e1015d390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e100c2c10 tx=0x7f4e0000be70 comp rx=0 tx=0).stop 2023-06-08T03:50:08.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 -- 172.21.15.156:0/4123193324 shutdown_connections 2023-06-08T03:50:08.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 --2- 172.21.15.156:0/4123193324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e1015d390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 --2- 172.21.15.156:0/4123193324 >> v2:172.21.15.79:3300/0 conn(0x7f4e10071070 0x7f4e100cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 -- 172.21.15.156:0/4123193324 >> 172.21.15.156:0/4123193324 conn(0x7f4e100b1d30 msgr2=0x7f4e1006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 -- 172.21.15.156:0/4123193324 shutdown_connections 2023-06-08T03:50:08.556 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.549+0000 7f4e1696e700 1 -- 172.21.15.156:0/4123193324 wait complete. 2023-06-08T03:50:08.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.550+0000 7f4e1696e700 1 Processor -- start 2023-06-08T03:50:08.558 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.550+0000 7f4e1696e700 1 -- start start 2023-06-08T03:50:08.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e1696e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e10071070 0x7f4e1014bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e1696e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 0x7f4e100c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e1696e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e100c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e1696e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1015a1c0 con 0x7f4e10071070 2023-06-08T03:50:08.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e1696e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e100b28e0 con 0x7f4e10158ec0 2023-06-08T03:50:08.559 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e1696e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e101595b0 con 0x7f4e100cc480 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e0f7fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 0x7f4e100c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e14f0b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e100c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e0f7fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 0x7f4e100c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:47782/0 (socket says 172.21.15.156:47782) 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e14f0b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e100c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:51376/0 (socket says 172.21.15.156:51376) 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e0f7fe700 1 -- 172.21.15.156:0/2214074411 learned_addr learned my addr 172.21.15.156:0/2214074411 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.551+0000 7f4e0ffff700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e10071070 0x7f4e1014bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e14f0b700 1 -- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 msgr2=0x7f4e100c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.560 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e14f0b700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 0x7f4e100c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e14f0b700 1 -- 172.21.15.156:0/2214074411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e10071070 msgr2=0x7f4e1014bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e14f0b700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e10071070 0x7f4e1014bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e14f0b700 1 -- 172.21.15.156:0/2214074411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0000b7f0 con 0x7f4e10158ec0 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0f7fe700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 0x7f4e100c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0ffff700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e10071070 0x7f4e1014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e14f0b700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e100c6640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800cb10 tx=0x7f4e0800ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800a9e0 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0800ab40 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e10159890 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08012420 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e10159e50 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e08012880 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.552+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f4e0801a3b0 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.553+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1015a330 con 0x7f4e10158ec0 2023-06-08T03:50:08.562 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.555+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4e0803ab00 con 0x7f4e10158ec0 2023-06-08T03:50:08.614 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:08.610+0000 7f90f4928200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-08T03:50:08.616 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:08.613+0000 7f7ea6d71200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-08T03:50:08.642 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.639+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "735d6453-9a2d-45cd-9877-3ad4133082d6", "id": 7} v 0) v1 -- 0x7f4e10073e90 con 0x7f4e10158ec0 2023-06-08T03:50:08.647 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.644+0000 7f4e0cff9700 1 -- 172.21.15.156:0/2214074411 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "735d6453-9a2d-45cd-9877-3ad4133082d6", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e08029070 con 0x7f4e10158ec0 2023-06-08T03:50:08.648 INFO:teuthology.orchestra.run.smithi156.stdout:7 2023-06-08T03:50:08.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 msgr2=0x7f4e100c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e100c6640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800cb10 tx=0x7f4e0800ce20 comp rx=0 tx=0).stop 2023-06-08T03:50:08.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 shutdown_connections 2023-06-08T03:50:08.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4e10158ec0 0x7f4e100c6640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e10071070 0x7f4e1014bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.649 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 --2- 172.21.15.156:0/2214074411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4e100cc480 0x7f4e100c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 >> 172.21.15.156:0/2214074411 conn(0x7f4e100b1d30 msgr2=0x7f4e10153480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 shutdown_connections 2023-06-08T03:50:08.650 INFO:teuthology.orchestra.run.smithi156.stderr:2023-06-08T03:50:08.645+0000 7f4e1696e700 1 -- 172.21.15.156:0/2214074411 wait complete. 2023-06-08T03:50:08.660 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-08T03:50:08.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-08T03:50:08.663 INFO:tasks.ceph.osd.0:Started 2023-06-08T03:50:08.663 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-08T03:50:08.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-08T03:50:08.666 INFO:tasks.ceph.osd.1:Started 2023-06-08T03:50:08.667 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-08T03:50:08.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-08T03:50:08.669 INFO:tasks.ceph.osd.2:Started 2023-06-08T03:50:08.670 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-08T03:50:08.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-08T03:50:08.672 INFO:tasks.ceph.osd.3:Started 2023-06-08T03:50:08.672 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-08T03:50:08.672 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-08T03:50:08.702 INFO:tasks.ceph.osd.4:Started 2023-06-08T03:50:08.703 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-08T03:50:08.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-08T03:50:08.706 INFO:tasks.ceph.osd.5:Started 2023-06-08T03:50:08.706 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-08T03:50:08.707 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-08T03:50:08.709 INFO:tasks.ceph.osd.6:Started 2023-06-08T03:50:08.710 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-08T03:50:08.711 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-08T03:50:08.713 INFO:tasks.ceph.osd.7:Started 2023-06-08T03:50:08.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:08.792 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:08.788+0000 7f90f4928200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-08T03:50:08.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.849+0000 7fa99c37f700 1 Processor -- start 2023-06-08T03:50:08.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.849+0000 7fa99c37f700 1 -- start start 2023-06-08T03:50:08.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99c37f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa99415d560 0x7fa994071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99c37f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9940cf5e0 0x7fa9940b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99c37f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa9940b20b0 0x7fa994069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99c37f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9940718a0 con 0x7fa9940cf5e0 2023-06-08T03:50:08.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99c37f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99406b5c0 con 0x7fa9940b20b0 2023-06-08T03:50:08.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99c37f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99406b2e0 con 0x7fa99415d560 2023-06-08T03:50:08.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9940cf5e0 0x7fa9940b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9940cf5e0 0x7fa9940b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33818/0 (socket says 172.21.15.79:33818) 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 -- 172.21.15.79:0/998172737 learned_addr learned my addr 172.21.15.79:0/998172737 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99a11b700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa99415d560 0x7fa994071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99a91c700 1 --2- 172.21.15.79:0/998172737 >> v2:172.21.15.156:3300/0 conn(0x7fa9940b20b0 0x7fa994069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 -- 172.21.15.79:0/998172737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa99415d560 msgr2=0x7fa994071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa99415d560 0x7fa994071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 -- 172.21.15.79:0/998172737 >> v2:172.21.15.156:3300/0 conn(0x7fa9940b20b0 msgr2=0x7fa994069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:08.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 --2- 172.21.15.79:0/998172737 >> v2:172.21.15.156:3300/0 conn(0x7fa9940b20b0 0x7fa994069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 -- 172.21.15.79:0/998172737 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994070880 con 0x7fa9940cf5e0 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99a11b700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa99415d560 0x7fa994071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa99991a700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940cf5e0 0x7fa9940b1b70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa994065aa0 tx=0x7fa984009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2141374afc260217 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa999119700 1 -- 172.21.15.79:0/998172737 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984011830 con 0x7fa9940cf5e0 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa999119700 1 -- 172.21.15.79:0/998172737 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984011990 con 0x7fa9940cf5e0 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.850+0000 7fa999119700 1 -- 172.21.15.79:0/998172737 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984011d20 con 0x7fa9940cf5e0 2023-06-08T03:50:08.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 -- 172.21.15.79:0/998172737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940cf5e0 msgr2=0x7fa9940b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940cf5e0 0x7fa9940b1b70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa994065aa0 tx=0x7fa984009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 -- 172.21.15.79:0/998172737 shutdown_connections 2023-06-08T03:50:08.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 --2- 172.21.15.79:0/998172737 >> v2:172.21.15.156:3300/0 conn(0x7fa9940b20b0 0x7fa994069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa99415d560 0x7fa994071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 --2- 172.21.15.79:0/998172737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940cf5e0 0x7fa9940b1b70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 -- 172.21.15.79:0/998172737 >> 172.21.15.79:0/998172737 conn(0x7fa9940b14a0 msgr2=0x7fa9940cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 -- 172.21.15.79:0/998172737 shutdown_connections 2023-06-08T03:50:08.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.851+0000 7fa99c37f700 1 -- 172.21.15.79:0/998172737 wait complete. 2023-06-08T03:50:08.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.853+0000 7fa99c37f700 1 Processor -- start 2023-06-08T03:50:08.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.853+0000 7fa99c37f700 1 -- start start 2023-06-08T03:50:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99c37f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 0x7fa994150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99c37f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9940cf5e0 0x7fa9941517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99c37f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa99415d560 0x7fa9941e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99c37f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9941512b0 con 0x7fa9940b20b0 2023-06-08T03:50:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99c37f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9941e2990 con 0x7fa99415d560 2023-06-08T03:50:08.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99c37f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9941e2be0 con 0x7fa9940cf5e0 2023-06-08T03:50:08.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99a11b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 0x7fa994150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99a11b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 0x7fa994150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33824/0 (socket says 172.21.15.79:33824) 2023-06-08T03:50:08.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99a11b700 1 -- 172.21.15.79:0/3751085420 learned_addr learned my addr 172.21.15.79:0/3751085420 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:08.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.854+0000 7fa99991a700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9940cf5e0 0x7fa9941517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a91c700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa99415d560 0x7fa9941e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:08.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a11b700 1 -- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9940cf5e0 msgr2=0x7fa9941517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a11b700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9940cf5e0 0x7fa9941517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a11b700 1 -- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa99415d560 msgr2=0x7fa9941e3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a11b700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa99415d560 0x7fa9941e3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a11b700 1 -- 172.21.15.79:0/3751085420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009550 con 0x7fa9940b20b0 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99a11b700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 0x7fa994150d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa99000e810 tx=0x7fa99000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99000c070 con 0x7fa9940b20b0 2023-06-08T03:50:08.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990004230 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9941e2ec0 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9900036a0 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9941e3480 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa990003b00 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.855+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa990003db0 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.856+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa994073e90 con 0x7fa9940b20b0 2023-06-08T03:50:08.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.858+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa99000f040 con 0x7fa9940b20b0 2023-06-08T03:50:08.931 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.927+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa994065b30 con 0x7fa9940b20b0 2023-06-08T03:50:08.932 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.928+0000 7fa98affd700 1 -- 172.21.15.79:0/3751085420 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fa9900134c0 con 0x7fa9940b20b0 2023-06-08T03:50:08.933 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":10,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:08.640471+0000","last_up_change":"0.000000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 msgr2=0x7fa994150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 0x7fa994150d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa99000e810 tx=0x7fa99000eb20 comp rx=0 tx=0).stop 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 shutdown_connections 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa99415d560 0x7fa9941e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940b20b0 0x7fa994150d70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 --2- 172.21.15.79:0/3751085420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa9940cf5e0 0x7fa9941517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:08.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 >> 172.21.15.79:0/3751085420 conn(0x7fa9940b14a0 msgr2=0x7fa9940cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:08.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 shutdown_connections 2023-06-08T03:50:08.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:08.929+0000 7fa99c37f700 1 -- 172.21.15.79:0/3751085420 wait complete. 2023-06-08T03:50:08.945 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-08T03:50:08.945 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-08T03:50:09.282 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T03:50:09.278+0000 7f75ae32b640 -1 Falling back to public interface 2023-06-08T03:50:09.291 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T03:50:09.287+0000 7fafa2451640 -1 Falling back to public interface 2023-06-08T03:50:09.308 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T03:50:09.304+0000 7f8d44b41640 -1 Falling back to public interface 2023-06-08T03:50:09.313 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T03:50:09.310+0000 7f4473576640 -1 Falling back to public interface 2023-06-08T03:50:09.314 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T03:50:09.311+0000 7ff7bb660640 -1 Falling back to public interface 2023-06-08T03:50:09.316 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T03:50:09.312+0000 7f4bda8a3640 -1 Falling back to public interface 2023-06-08T03:50:09.324 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T03:50:09.321+0000 7fbbafc33640 -1 Falling back to public interface 2023-06-08T03:50:09.330 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T03:50:09.327+0000 7f097aad9640 -1 Falling back to public interface 2023-06-08T03:50:09.348 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:09.345+0000 7f7ea6d71200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-08T03:50:09.426 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:09.423+0000 7f7ea6d71200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-08T03:50:09.490 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:09.486+0000 7f90f4928200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-08T03:50:09.564 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:09.560+0000 7f7ea6d71200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-08T03:50:09.566 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:09.563+0000 7f90f4928200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-08T03:50:09.706 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:09.702+0000 7f90f4928200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-08T03:50:09.753 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:09.800 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:09.796+0000 7f7ea6d71200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-08T03:50:09.865 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.861+0000 7fadfc25d700 1 Processor -- start 2023-06-08T03:50:09.866 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.862+0000 7fadfc25d700 1 -- start start 2023-06-08T03:50:09.868 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.862+0000 7fadfc25d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fadf4071310 0x7fadf40cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:09.868 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.862+0000 7fadfc25d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40ce740 con 0x7fadf40afa70 2023-06-08T03:50:09.868 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.862+0000 7fadfc25d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40cea20 con 0x7fadf4159e40 2023-06-08T03:50:09.868 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.862+0000 7fadfc25d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf40ced00 con 0x7fadf4071310 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadfa7fa700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fadf4071310 0x7fadf40cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadf97f8700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fadf40afa70 0x7fadf4070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57306/0 (socket says 172.21.15.79:57306) 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadf97f8700 1 -- 172.21.15.79:0/2315934802 learned_addr learned my addr 172.21.15.79:0/2315934802 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadfa7fa700 1 -- 172.21.15.79:0/2315934802 >> v1:172.21.15.156:6789/0 conn(0x7fadf4159e40 legacy=0x7fadf406c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadfa7fa700 1 -- 172.21.15.79:0/2315934802 >> v1:172.21.15.79:6789/0 conn(0x7fadf40afa70 legacy=0x7fadf4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadf8ff7700 1 -- 172.21.15.79:0/2315934802 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226818774 0 0) 0x7fadf40ce740 con 0x7fadf40afa70 2023-06-08T03:50:09.869 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadfa7fa700 1 -- 172.21.15.79:0/2315934802 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40cefe0 con 0x7fadf4071310 2023-06-08T03:50:09.870 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadfa7fa700 1 --2- 172.21.15.79:0/2315934802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf4071310 0x7fadf40cb520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fadec009d90 tx=0x7fadec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9e888739b39e63 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:09.870 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadf8ff7700 1 -- 172.21.15.79:0/2315934802 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00e070 con 0x7fadf4071310 2023-06-08T03:50:09.870 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadf8ff7700 1 -- 172.21.15.79:0/2315934802 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec00bbd0 con 0x7fadf4071310 2023-06-08T03:50:09.870 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.863+0000 7fadf8ff7700 1 -- 172.21.15.79:0/2315934802 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec012430 con 0x7fadf4071310 2023-06-08T03:50:09.870 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 -- 172.21.15.79:0/2315934802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf4071310 msgr2=0x7fadf40cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:09.871 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 --2- 172.21.15.79:0/2315934802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf4071310 0x7fadf40cb520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fadec009d90 tx=0x7fadec00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:09.871 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 -- 172.21.15.79:0/2315934802 shutdown_connections 2023-06-08T03:50:09.871 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 --2- 172.21.15.79:0/2315934802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf4071310 0x7fadf40cb520 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:09.871 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 -- 172.21.15.79:0/2315934802 >> 172.21.15.79:0/2315934802 conn(0x7fadf4069db0 msgr2=0x7fadf406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:09.872 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 -- 172.21.15.79:0/2315934802 shutdown_connections 2023-06-08T03:50:09.872 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.864+0000 7fadfc25d700 1 -- 172.21.15.79:0/2315934802 wait complete. 2023-06-08T03:50:09.875 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.865+0000 7fadfc25d700 1 Processor -- start 2023-06-08T03:50:09.875 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.865+0000 7fadfc25d700 1 -- start start 2023-06-08T03:50:09.875 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf4071310 0x7fadf413d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:09.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 0x7fadf40c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:09.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf4159e40 0x7fadf4162d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:09.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b3050 con 0x7fadf40afa70 2023-06-08T03:50:09.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b2630 con 0x7fadf4159e40 2023-06-08T03:50:09.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40b32a0 con 0x7fadf4071310 2023-06-08T03:50:09.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 0x7fadf40c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 0x7fadf40c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:33836/0 (socket says 172.21.15.79:33836) 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 -- 172.21.15.79:0/3729513030 learned_addr learned my addr 172.21.15.79:0/3729513030 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfa7fa700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf4159e40 0x7fadf4162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf9ff9700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf4071310 0x7fadf413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 -- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf4071310 msgr2=0x7fadf413d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf4071310 0x7fadf413d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:09.877 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 -- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf4159e40 msgr2=0x7fadf4162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf4159e40 0x7fadf4162d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 -- 172.21.15.79:0/3729513030 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec009a70 con 0x7fadf40afa70 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadf97f8700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 0x7fadf40c5910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c520 tx=0x7fadf000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0004c80 con 0x7fadf40afa70 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0004e00 con 0x7fadf40afa70 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007310 con 0x7fadf40afa70 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.866+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf40b3580 con 0x7fadf40afa70 2023-06-08T03:50:09.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.867+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf413deb0 con 0x7fadf40afa70 2023-06-08T03:50:09.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.867+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fadf001d3f0 con 0x7fadf40afa70 2023-06-08T03:50:09.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.867+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fadf0013a90 con 0x7fadf40afa70 2023-06-08T03:50:09.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.867+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf40b3aa0 con 0x7fadf40afa70 2023-06-08T03:50:09.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.869+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fadf40b3aa0 con 0x7fadf40afa70 2023-06-08T03:50:09.879 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:09.871+0000 7f7ea6d71200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-08T03:50:09.942 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.938+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadf4073e90 con 0x7fadf40afa70 2023-06-08T03:50:09.943 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.939+0000 7fade67fc700 1 -- 172.21.15.79:0/3729513030 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fadf0013de0 con 0x7fadf40afa70 2023-06-08T03:50:09.944 INFO:teuthology.misc.health.smithi079.stdout: 2023-06-08T03:50:09.944 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":10,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:08.640471+0000","last_up_change":"0.000000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:09.945 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 msgr2=0x7fadf40c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:09.946 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 0x7fadf40c5910 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fadf000c520 tx=0x7fadf000c830 comp rx=0 tx=0).stop 2023-06-08T03:50:09.946 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 shutdown_connections 2023-06-08T03:50:09.946 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadf4159e40 0x7fadf4162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:09.947 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadf4071310 0x7fadf413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:09.947 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 --2- 172.21.15.79:0/3729513030 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fadf40afa70 0x7fadf40c5910 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:09.947 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 >> 172.21.15.79:0/3729513030 conn(0x7fadf4069db0 msgr2=0x7fadf406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:09.947 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 shutdown_connections 2023-06-08T03:50:09.948 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:09.940+0000 7fadfc25d700 1 -- 172.21.15.79:0/3729513030 wait complete. 2023-06-08T03:50:09.948 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:09.943+0000 7f90f4928200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-08T03:50:09.954 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-08T03:50:10.022 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:10.018+0000 7f90f4928200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-08T03:50:10.148 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:10.145+0000 7f7ea6d71200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-08T03:50:10.302 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:10.298+0000 7f90f4928200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-08T03:50:10.366 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:10.363+0000 7f7ea6d71200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-08T03:50:10.520 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:10.516+0000 7f90f4928200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-08T03:50:10.524 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:10.521+0000 7f7ea6d71200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-08T03:50:10.638 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:10.635+0000 7f7ea6d71200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-08T03:50:10.676 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:10.672+0000 7f90f4928200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-08T03:50:10.790 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:10.786+0000 7f90f4928200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-08T03:50:10.963 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:10.960+0000 7f7ea6d71200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-08T03:50:11.064 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:11.061+0000 7f7ea6d71200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-08T03:50:11.107 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:11.103+0000 7f90f4928200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-08T03:50:11.205 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:11.201+0000 7f90f4928200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-08T03:50:11.252 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:11.249+0000 7f7ea6d71200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-08T03:50:11.389 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:11.385+0000 7f90f4928200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-08T03:50:11.849 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:11.846+0000 7f7ea6d71200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-08T03:50:11.991 INFO:tasks.ceph.mgr.y.smithi079.stderr:2023-06-08T03:50:11.987+0000 7f90f4928200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-08T03:50:12.865 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T03:50:12.862+0000 7f4473576640 -1 osd.4 0 log_to_monitors true 2023-06-08T03:50:12.869 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T03:50:12.865+0000 7fafa2451640 -1 osd.2 0 log_to_monitors true 2023-06-08T03:50:12.870 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T03:50:12.867+0000 7ff7bb660640 -1 osd.5 0 log_to_monitors true 2023-06-08T03:50:12.879 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T03:50:12.875+0000 7f8d44b41640 -1 osd.1 0 log_to_monitors true 2023-06-08T03:50:12.880 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:12.876+0000 7f7e5ef9d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-08T03:50:12.880 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-06-08T03:50:12.877+0000 7f7e5ef9d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-08T03:50:12.885 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T03:50:12.882+0000 7f097aad9640 -1 osd.6 0 log_to_monitors true 2023-06-08T03:50:12.886 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T03:50:12.883+0000 7fbbafc33640 -1 osd.7 0 log_to_monitors true 2023-06-08T03:50:12.887 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T03:50:12.883+0000 7f75ae32b640 -1 osd.0 0 log_to_monitors true 2023-06-08T03:50:12.888 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T03:50:12.885+0000 7f4bda8a3640 -1 osd.3 0 log_to_monitors true 2023-06-08T03:50:15.217 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T03:50:15.214+0000 7f446792e700 -1 osd.4 0 waiting for initial osdmap 2023-06-08T03:50:15.246 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T03:50:15.241+0000 7f096f692700 -1 osd.6 0 waiting for initial osdmap 2023-06-08T03:50:15.247 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T03:50:15.243+0000 7ff7afa18700 -1 osd.5 0 waiting for initial osdmap 2023-06-08T03:50:15.253 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T03:50:15.250+0000 7f445f508700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:15.259 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T03:50:15.256+0000 7f0966a6b700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:15.261 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T03:50:15.258+0000 7ff7a75f2700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:15.288 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T03:50:15.285+0000 7fbba3feb700 -1 osd.7 0 waiting for initial osdmap 2023-06-08T03:50:15.301 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T03:50:15.298+0000 7fbb9bbc5700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:15.982 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T03:50:15.977+0000 7f75a26e3700 -1 osd.0 0 waiting for initial osdmap 2023-06-08T03:50:15.983 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T03:50:15.977+0000 7faf96809700 -1 osd.2 0 waiting for initial osdmap 2023-06-08T03:50:15.996 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T03:50:15.993+0000 7f759a2bd700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:15.999 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T03:50:15.997+0000 7faf8e3e3700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:16.014 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T03:50:16.011+0000 7f4bcec5b700 -1 osd.3 0 waiting for initial osdmap 2023-06-08T03:50:16.017 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T03:50:16.014+0000 7f8d38ef9700 -1 osd.1 0 waiting for initial osdmap 2023-06-08T03:50:16.027 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T03:50:16.024+0000 7f8d30ad3700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:16.027 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T03:50:16.024+0000 7f4bc6835700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-08T03:50:16.761 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:16.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.873+0000 7f5dcdafb700 1 Processor -- start 2023-06-08T03:50:16.876 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.873+0000 7f5dcdafb700 1 -- start start 2023-06-08T03:50:16.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dcdafb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc8158790 0x7f5dc80c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:16.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dcdafb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc80c9a30 con 0x7f5dc80cdbe0 2023-06-08T03:50:16.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dcdafb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc815d050 con 0x7f5dc80b0620 2023-06-08T03:50:16.878 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dcdafb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc80ce110 con 0x7f5dc8158790 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc8158790 0x7f5dc80c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc6ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5dc80cdbe0 0x7f5dc80aff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57514/0 (socket says 172.21.15.79:57514) 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc6ffd700 1 -- 172.21.15.79:0/245696280 learned_addr learned my addr 172.21.15.79:0/245696280 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc77fe700 1 -- 172.21.15.79:0/245696280 >> v1:172.21.15.156:6789/0 conn(0x7f5dc80b0620 legacy=0x7f5dc80cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc77fe700 1 -- 172.21.15.79:0/245696280 >> v1:172.21.15.79:6789/0 conn(0x7f5dc80cdbe0 legacy=0x7f5dc80aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc77fe700 1 -- 172.21.15.79:0/245696280 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8071c10 con 0x7f5dc8158790 2023-06-08T03:50:16.879 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.874+0000 7f5dc67fc700 1 -- 172.21.15.79:0/245696280 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294099665 0 0) 0x7f5db0001110 con 0x7f5dc80cdbe0 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dc77fe700 1 --2- 172.21.15.79:0/245696280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc80c94f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009840 tx=0x7f5db8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b691a1294b939af5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dc67fc700 1 -- 172.21.15.79:0/245696280 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8011830 con 0x7f5dc8158790 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dc67fc700 1 -- 172.21.15.79:0/245696280 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db8011990 con 0x7f5dc8158790 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dc67fc700 1 -- 172.21.15.79:0/245696280 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8011d20 con 0x7f5dc8158790 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 -- 172.21.15.79:0/245696280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 msgr2=0x7f5dc80c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/245696280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc80c94f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009840 tx=0x7f5db8009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 -- 172.21.15.79:0/245696280 shutdown_connections 2023-06-08T03:50:16.880 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/245696280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc80c94f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:16.881 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 -- 172.21.15.79:0/245696280 >> 172.21.15.79:0/245696280 conn(0x7f5dc8069db0 msgr2=0x7f5dc806a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:16.881 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 -- 172.21.15.79:0/245696280 shutdown_connections 2023-06-08T03:50:16.881 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.875+0000 7f5dcdafb700 1 -- 172.21.15.79:0/245696280 wait complete. 2023-06-08T03:50:16.883 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.876+0000 7f5dcdafb700 1 Processor -- start 2023-06-08T03:50:16.883 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 -- start start 2023-06-08T03:50:16.883 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc80b0620 0x7f5dc80b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc80cdbe0 0x7f5dc815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc8147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc814a060 con 0x7f5dc8158790 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80c3f80 con 0x7f5dc80cdbe0 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dcdafb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc815a3c0 con 0x7f5dc80b0620 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc8147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc8147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34046/0 (socket says 172.21.15.79:34046) 2023-06-08T03:50:16.884 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.877+0000 7f5dc7fff700 1 -- 172.21.15.79:0/2267561877 learned_addr learned my addr 172.21.15.79:0/2267561877 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc6ffd700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc80cdbe0 0x7f5dc815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc7fff700 1 -- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc80b0620 msgr2=0x7f5dc80b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc77fe700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc80b0620 0x7f5dc80b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc7fff700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc80b0620 0x7f5dc80b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc7fff700 1 -- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc80cdbe0 msgr2=0x7f5dc815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc7fff700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc80cdbe0 0x7f5dc815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:16.885 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc7fff700 1 -- 172.21.15.79:0/2267561877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db80094a0 con 0x7f5dc8158790 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc6ffd700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc80cdbe0 0x7f5dc815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc77fe700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc80b0620 0x7f5dc80b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dc7fff700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc8147370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00021c0 tx=0x7f5dc00021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000cea0 con 0x7f5dc8158790 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0003c00 con 0x7f5dc8158790 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc815a6a0 con 0x7f5dc8158790 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00050d0 con 0x7f5dc8158790 2023-06-08T03:50:16.886 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.878+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc80b3a80 con 0x7f5dc8158790 2023-06-08T03:50:16.887 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.879+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc815aea0 con 0x7f5dc8158790 2023-06-08T03:50:16.887 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.879+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f5dc0005230 con 0x7f5dc8158790 2023-06-08T03:50:16.887 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.880+0000 7f5dbffff700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5da806ac50 0x7f5da806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:16.887 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.880+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5dc008b570 con 0x7f5dc8158790 2023-06-08T03:50:16.887 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.880+0000 7f5dc77fe700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5da806ac50 0x7f5da806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:16.887 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.880+0000 7f5dc77fe700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5da806ac50 0x7f5da806d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:16.888 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:16.884+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5dc815aea0 con 0x7f5dc8158790 2023-06-08T03:50:17.079 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.076+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dc815aea0 con 0x7f5dc8158790 2023-06-08T03:50:17.080 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.077+0000 7f5dbffff700 1 -- 172.21.15.79:0/2267561877 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10884 (secure 0 0 0) 0x7f5dc815aea0 con 0x7f5dc8158790 2023-06-08T03:50:17.080 INFO:teuthology.misc.health.smithi079.stdout: 2023-06-08T03:50:17.081 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":13,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:16.138854+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:17.082 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5da806ac50 msgr2=0x7f5da806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.082 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5da806ac50 0x7f5da806d100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db801c040 comp rx=0 tx=0).stop 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 msgr2=0x7f5dc8147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc8147370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00021c0 tx=0x7f5dc00021f0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 shutdown_connections 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8158790 0x7f5dc8147370 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5da806ac50 0x7f5da806d100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc80b0620 0x7f5dc80b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.083 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 --2- 172.21.15.79:0/2267561877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc80cdbe0 0x7f5dc815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.084 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 >> 172.21.15.79:0/2267561877 conn(0x7f5dc8069db0 msgr2=0x7f5dc80cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:17.084 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 shutdown_connections 2023-06-08T03:50:17.084 INFO:teuthology.misc.health.smithi079.stderr:2023-06-08T03:50:17.079+0000 7f5dcdafb700 1 -- 172.21.15.79:0/2267561877 wait complete. 2023-06-08T03:50:17.093 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-08T03:50:17.093 INFO:tasks.ceph:Creating RBD pool 2023-06-08T03:50:17.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-08T03:50:17.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.211+0000 7f75d0f23700 1 Processor -- start 2023-06-08T03:50:17.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.211+0000 7f75d0f23700 1 -- start start 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.212+0000 7f75d0f23700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 0x7f75cc0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.212+0000 7f75d0f23700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0cae50 con 0x7f75cc0cd080 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.212+0000 7f75d0f23700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0afee0 con 0x7f75cc0d0060 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.212+0000 7f75d0f23700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0cbea0 con 0x7f75cc0cd4f0 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.212+0000 7f75c9d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f75cc0cd080 0x7f75cc0ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57534/0 (socket says 172.21.15.79:57534) 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.212+0000 7f75c9d9b700 1 -- 172.21.15.79:0/129965696 learned_addr learned my addr 172.21.15.79:0/129965696 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927424345 0 0) 0x7f75cc0cae50 con 0x7f75cc0cd080 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 0x7f75cc0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b40035b0 con 0x7f75cc0cd080 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855985855 0 0) 0x7f75b40035b0 con 0x7f75cc0cd080 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 msgr2=0x7f75cc0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 0x7f75cc0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 >> v1:172.21.15.156:6789/0 conn(0x7f75cc0d0060 legacy=0x7f75cc0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc15d930 con 0x7f75cc0cd080 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f75bc001670 con 0x7f75cc0cd080 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 >> v1:172.21.15.79:6789/0 conn(0x7f75cc0cd080 legacy=0x7f75cc0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75b4003a30 0x7f75b4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 0x7f75b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400bac0 con 0x7f75b4004d90 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400bc00 con 0x7f75b4003a30 2023-06-08T03:50:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b400bdc0 con 0x7f75cc0cd4f0 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75ca59c700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75b4003a30 0x7f75b4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75c9d9b700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 0x7f75b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 -- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 msgr2=0x7f75b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0cd4f0 0x7f75b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 -- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75b4003a30 msgr2=0x7f75b4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75b4003a30 0x7f75b4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.213+0000 7f75cad9d700 1 -- 172.21.15.79:0/129965696 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40035b0 con 0x7f75b4004d90 2023-06-08T03:50:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75cad9d700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75b4005200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009fc0 tx=0x7f75b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1444e32b1c2901df server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400ec30 con 0x7f75b4004d90 2023-06-08T03:50:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c400f040 con 0x7f75b4004d90 2023-06-08T03:50:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75c959a700 1 -- 172.21.15.79:0/129965696 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4014480 con 0x7f75b4004d90 2023-06-08T03:50:17.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75d0f23700 1 -- 172.21.15.79:0/129965696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 msgr2=0x7f75b4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75d0f23700 1 --2- 172.21.15.79:0/129965696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75b4005200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009fc0 tx=0x7f75b4005880 comp rx=0 tx=0).stop 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75cad9d700 1 -- 172.21.15.79:0/129965696 reap_dead start 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75d0f23700 1 -- 172.21.15.79:0/129965696 shutdown_connections 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75d0f23700 1 -- 172.21.15.79:0/129965696 >> 172.21.15.79:0/129965696 conn(0x7f75cc06e1e0 msgr2=0x7f75cc06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75d0f23700 1 -- 172.21.15.79:0/129965696 shutdown_connections 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.214+0000 7f75d0f23700 1 -- 172.21.15.79:0/129965696 wait complete. 2023-06-08T03:50:17.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 Processor -- start 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 -- start start 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75cc154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0b19d0 0x7f75cc0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1598d0 0x7f75cc159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc155190 con 0x7f75b4004d90 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc15a280 con 0x7f75cc1598d0 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75d0f23700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0b2770 con 0x7f75cc0b19d0 2023-06-08T03:50:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75ca59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75cc154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75ca59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75cc154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34070/0 (socket says 172.21.15.79:34070) 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.216+0000 7f75ca59c700 1 -- 172.21.15.79:0/2194529068 learned_addr learned my addr 172.21.15.79:0/2194529068 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75cad9d700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1598d0 0x7f75cc159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75c9d9b700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0b19d0 0x7f75cc0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75ca59c700 1 -- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0b19d0 msgr2=0x7f75cc0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75ca59c700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0b19d0 0x7f75cc0b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75ca59c700 1 -- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1598d0 msgr2=0x7f75cc159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75ca59c700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1598d0 0x7f75cc159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75ca59c700 1 -- 172.21.15.79:0/2194529068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc06ae60 con 0x7f75b4004d90 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75cad9d700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1598d0 0x7f75cc159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75ca59c700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75cc154c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75c000f820 tx=0x7f75c000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00052e0 con 0x7f75b4004d90 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0005440 con 0x7f75b4004d90 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00038c0 con 0x7f75b4004d90 2023-06-08T03:50:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc0b2490 con 0x7f75b4004d90 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.217+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc06b390 con 0x7f75b4004d90 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.218+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc073e90 con 0x7f75b4004d90 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.218+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f75c0003a20 con 0x7f75b4004d90 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.219+0000 7f75baffd700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75ac06ac50 0x7f75ac06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.219+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f75c0088ba0 con 0x7f75b4004d90 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.219+0000 7f75c9d9b700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75ac06ac50 0x7f75ac06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.219+0000 7f75c9d9b700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75ac06ac50 0x7f75ac06d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0cce60 tx=0x7f75bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.223+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75c0054360 con 0x7f75b4004d90 2023-06-08T03:50:17.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:17.412+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f75cc0b3640 con 0x7f75b4004d90 2023-06-08T03:50:18.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.159+0000 7f75baffd700 1 -- 172.21.15.79:0/2194529068 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f75c0057980 con 0x7f75b4004d90 2023-06-08T03:50:18.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75ac06ac50 msgr2=0x7f75ac06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:18.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75ac06ac50 0x7f75ac06d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0cce60 tx=0x7f75bc009040 comp rx=0 tx=0).stop 2023-06-08T03:50:18.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 msgr2=0x7f75cc154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:18.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75cc154c50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f75c000f820 tx=0x7f75c000fbe0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 shutdown_connections 2023-06-08T03:50:18.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1598d0 0x7f75cc159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b4004d90 0x7f75cc154c50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75ac06ac50 0x7f75ac06d100 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0cce60 tx=0x7f75bc009040 comp rx=0 tx=0).stop 2023-06-08T03:50:18.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.161+0000 7f75d0f23700 1 --2- 172.21.15.79:0/2194529068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75cc0b19d0 0x7f75cc0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.162+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 >> 172.21.15.79:0/2194529068 conn(0x7f75cc06e1e0 msgr2=0x7f75cc0ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:18.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.162+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 shutdown_connections 2023-06-08T03:50:18.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.162+0000 7f75d0f23700 1 -- 172.21.15.79:0/2194529068 wait complete. 2023-06-08T03:50:18.167 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'rbd' created 2023-06-08T03:50:18.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-08T03:50:18.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.300+0000 7f4acc396700 1 Processor -- start 2023-06-08T03:50:18.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.300+0000 7f4acc396700 1 -- start start 2023-06-08T03:50:18.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4acc396700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4ac40cf170 0x7f4ac40ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:18.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4acc396700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cd060 0x7f4ac40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:18.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4acc396700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac40ce760 con 0x7f4ac40cf680 2023-06-08T03:50:18.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4acc396700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac40cda20 con 0x7f4ac40cf170 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4acc396700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40caf40 con 0x7f4ac40cd060 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4aca132700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4ac40cf170 0x7f4ac40ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4aca933700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cd060 0x7f4ac40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4ac9931700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4ac40cf680 0x7f4ac40af7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:59740/0 (socket says 172.21.15.79:59740) 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.301+0000 7f4ac9931700 1 -- 172.21.15.79:0/820998551 learned_addr learned my addr 172.21.15.79:0/820998551 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4aca933700 1 -- 172.21.15.79:0/820998551 >> v2:172.21.15.156:3300/0 conn(0x7f4ac40cf170 msgr2=0x7f4ac40ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:18.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4aca933700 1 --2- 172.21.15.79:0/820998551 >> v2:172.21.15.156:3300/0 conn(0x7f4ac40cf170 0x7f4ac40ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4aca933700 1 -- 172.21.15.79:0/820998551 >> v1:172.21.15.156:6789/0 conn(0x7f4ac40cf680 legacy=0x7f4ac40af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4aca933700 1 -- 172.21.15.79:0/820998551 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40cc690 con 0x7f4ac40cd060 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4ac9130700 1 -- 172.21.15.79:0/820998551 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956495034 0 0) 0x7f4ac40ce760 con 0x7f4ac40cf680 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4aca132700 1 --2- 172.21.15.79:0/820998551 >> v2:172.21.15.156:3300/0 conn(0x7f4ac40cf170 0x7f4ac40ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4aca933700 1 --2- 172.21.15.79:0/820998551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cd060 0x7f4ac40cd4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b0a2a8e84e4769 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4ac9130700 1 -- 172.21.15.79:0/820998551 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00e070 con 0x7f4ac40cd060 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4ac9130700 1 -- 172.21.15.79:0/820998551 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00bb20 con 0x7f4ac40cd060 2023-06-08T03:50:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4ac9130700 1 -- 172.21.15.79:0/820998551 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc012430 con 0x7f4ac40cd060 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4acc396700 1 -- 172.21.15.79:0/820998551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cd060 msgr2=0x7f4ac40cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.302+0000 7f4acc396700 1 --2- 172.21.15.79:0/820998551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cd060 0x7f4ac40cd4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.303+0000 7f4acc396700 1 -- 172.21.15.79:0/820998551 shutdown_connections 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.303+0000 7f4acc396700 1 --2- 172.21.15.79:0/820998551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cd060 0x7f4ac40cd4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.303+0000 7f4acc396700 1 --2- 172.21.15.79:0/820998551 >> v2:172.21.15.156:3300/0 conn(0x7f4ac40cf170 0x7f4ac40ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.303+0000 7f4acc396700 1 -- 172.21.15.79:0/820998551 >> 172.21.15.79:0/820998551 conn(0x7f4ac406e1e0 msgr2=0x7f4ac406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.303+0000 7f4acc396700 1 -- 172.21.15.79:0/820998551 shutdown_connections 2023-06-08T03:50:18.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.303+0000 7f4acc396700 1 -- 172.21.15.79:0/820998551 wait complete. 2023-06-08T03:50:18.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.304+0000 7f4acc396700 1 Processor -- start 2023-06-08T03:50:18.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.304+0000 7f4acc396700 1 -- start start 2023-06-08T03:50:18.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.304+0000 7f4acc396700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac40cd060 0x7f4ac4146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:18.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.304+0000 7f4acc396700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cf170 0x7f4ac40c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:18.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4acc396700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 0x7f4ac41490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:18.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4acc396700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40c67a0 con 0x7f4ac40cf680 2023-06-08T03:50:18.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4acc396700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac4149c30 con 0x7f4ac40cd060 2023-06-08T03:50:18.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4acc396700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac4149f10 con 0x7f4ac40cf170 2023-06-08T03:50:18.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 0x7f4ac41490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:18.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 0x7f4ac41490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34110/0 (socket says 172.21.15.79:34110) 2023-06-08T03:50:18.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 -- 172.21.15.79:0/1824421632 learned_addr learned my addr 172.21.15.79:0/1824421632 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:18.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca132700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac40cd060 0x7f4ac4146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:18.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4ac9931700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cf170 0x7f4ac40c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:18.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 -- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cf170 msgr2=0x7f4ac40c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:18.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cf170 0x7f4ac40c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 -- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac40cd060 msgr2=0x7f4ac4146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:18.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac40cd060 0x7f4ac4146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:18.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 -- 172.21.15.79:0/1824421632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40094a0 con 0x7f4ac40cf680 2023-06-08T03:50:18.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca132700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac40cd060 0x7f4ac4146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:18.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4aca933700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 0x7f4ac41490c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4abc014040 tx=0x7f4abc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:18.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc012430 con 0x7f4ac40cf680 2023-06-08T03:50:18.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc020070 con 0x7f4ac40cf680 2023-06-08T03:50:18.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc004280 con 0x7f4ac40cf680 2023-06-08T03:50:18.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc009a70 con 0x7f4ac40cf680 2023-06-08T03:50:18.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.305+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac40b46e0 con 0x7f4ac40cf680 2023-06-08T03:50:18.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.306+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac40c6950 con 0x7f4ac40cf680 2023-06-08T03:50:18.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.307+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f4abc0043e0 con 0x7f4ac40cf680 2023-06-08T03:50:18.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.307+0000 7f4abaffd700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4aa806ac50 0x7f4aa806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:18.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.307+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f4abc022070 con 0x7f4ac40cf680 2023-06-08T03:50:18.321 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.307+0000 7f4aca132700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4aa806ac50 0x7f4aa806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:18.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.308+0000 7f4aca132700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4aa806ac50 0x7f4aa806d100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009c50 tx=0x7f4ab4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:18.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.311+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4ac40c6950 con 0x7f4ac40cf680 2023-06-08T03:50:18.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:18.499+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ac4073e90 con 0x7f4ac40cf680 2023-06-08T03:50:19.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.163+0000 7f4abaffd700 1 -- 172.21.15.79:0/1824421632 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4abc02c0c0 con 0x7f4ac40cf680 2023-06-08T03:50:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4aa806ac50 msgr2=0x7f4aa806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4aa806ac50 0x7f4aa806d100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009c50 tx=0x7f4ab4008040 comp rx=0 tx=0).stop 2023-06-08T03:50:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 msgr2=0x7f4ac41490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 0x7f4ac41490c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4abc014040 tx=0x7f4abc004ad0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 shutdown_connections 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ac40cf680 0x7f4ac41490c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac40cd060 0x7f4ac4146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4aa806ac50 0x7f4aa806d100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 --2- 172.21.15.79:0/1824421632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ac40cf170 0x7f4ac40c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 >> 172.21.15.79:0/1824421632 conn(0x7f4ac406e1e0 msgr2=0x7f4ac40b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:19.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 shutdown_connections 2023-06-08T03:50:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.168+0000 7f4acc396700 1 -- 172.21.15.79:0/1824421632 wait complete. 2023-06-08T03:50:19.173 INFO:teuthology.orchestra.run.smithi079.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-08T03:50:19.186 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-08T03:50:19.187 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-08T03:50:19.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-08T03:50:19.189 INFO:tasks.ceph.mds.a:Started 2023-06-08T03:50:19.189 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-08T03:50:19.189 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-08T03:50:19.191 INFO:tasks.ceph.mds.c:Started 2023-06-08T03:50:19.191 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-08T03:50:19.191 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-08T03:50:19.193 INFO:tasks.ceph.mds.b:Started 2023-06-08T03:50:19.194 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-08T03:50:19.194 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-08T03:50:19.196 INFO:tasks.ceph.mds.d:Started 2023-06-08T03:50:19.196 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-08T03:50:19.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:19.282 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2023-06-08T03:50:19.283 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2023-06-08T03:50:19.284 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-06-08T03:50:19.284 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-06-08T03:50:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.336+0000 7f601d4d3700 1 Processor -- start 2023-06-08T03:50:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.336+0000 7f601d4d3700 1 -- start start 2023-06-08T03:50:19.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f601d4d3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 0x7f60180b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f601d4d3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60180b0750 con 0x7f601815a640 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f601d4d3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60180ceeb0 con 0x7f601815aa10 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f601d4d3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180b09a0 con 0x7f60180afc90 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f6016ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f601815a640 0x7f60180ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57614/0 (socket says 172.21.15.79:57614) 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f6016ffd700 1 -- 172.21.15.79:0/2126837441 learned_addr learned my addr 172.21.15.79:0/2126837441 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f60177fe700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 0x7f60180b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498196466 0 0) 0x7f60180b0750 con 0x7f601815a640 2023-06-08T03:50:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.337+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60000035b0 con 0x7f601815a640 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980285257 0 0) 0x7f60000035b0 con 0x7f601815a640 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 msgr2=0x7f60180b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 0x7f60180b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 >> v1:172.21.15.156:6789/0 conn(0x7f601815aa10 legacy=0x7f60180af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180b0c80 con 0x7f601815a640 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f6008001670 con 0x7f601815a640 2023-06-08T03:50:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 >> v1:172.21.15.79:6789/0 conn(0x7f601815a640 legacy=0x7f60180ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 0x7f6000003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000003fe0 0x7f6000004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6000004d90 0x7f6000005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180b09a0 con 0x7f6000004d90 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600000bac0 con 0x7f6000003fe0 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600000bc80 con 0x7f60180afc90 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6000004d90 0x7f6000005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6016ffd700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 0x7f6000003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60167fc700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000003fe0 0x7f6000004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 -- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 msgr2=0x7f6000003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60180afc90 0x7f6000003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 -- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000003fe0 msgr2=0x7f6000004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000003fe0 0x7f6000004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 -- 172.21.15.79:0/2126837441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000035b0 con 0x7f6000004d90 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f60177fe700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6000004d90 0x7f6000005200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6018072370 tx=0x7f6000005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f40f800c7b2a230 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:19.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000ebd0 con 0x7f6000004d90 2023-06-08T03:50:19.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601000f040 con 0x7f6000004d90 2023-06-08T03:50:19.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.338+0000 7f6015ffb700 1 -- 172.21.15.79:0/2126837441 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010014480 con 0x7f6000004d90 2023-06-08T03:50:19.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.339+0000 7f601d4d3700 1 -- 172.21.15.79:0/2126837441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6000004d90 msgr2=0x7f6000005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.339+0000 7f601d4d3700 1 --2- 172.21.15.79:0/2126837441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6000004d90 0x7f6000005200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6018072370 tx=0x7f6000005880 comp rx=0 tx=0).stop 2023-06-08T03:50:19.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.339+0000 7f60177fe700 1 -- 172.21.15.79:0/2126837441 reap_dead start 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.339+0000 7f601d4d3700 1 -- 172.21.15.79:0/2126837441 shutdown_connections 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.339+0000 7f601d4d3700 1 -- 172.21.15.79:0/2126837441 >> 172.21.15.79:0/2126837441 conn(0x7f601806a670 msgr2=0x7f6018099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.340+0000 7f601d4d3700 1 -- 172.21.15.79:0/2126837441 shutdown_connections 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.340+0000 7f601d4d3700 1 -- 172.21.15.79:0/2126837441 wait complete. 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.342+0000 7f601d4d3700 1 Processor -- start 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.342+0000 7f601d4d3700 1 -- start start 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000004d90 0x7f6018150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 0x7f60180b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6018155f50 0x7f60181563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018151150 con 0x7f60180b3650 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018156900 con 0x7f6000004d90 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180b3e80 con 0x7f6018155f50 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 0x7f60180b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 0x7f60180b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34152/0 (socket says 172.21.15.79:34152) 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 -- 172.21.15.79:0/1184565797 learned_addr learned my addr 172.21.15.79:0/1184565797 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:19.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60177fe700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6018155f50 0x7f60181563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f6016ffd700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000004d90 0x7f6018150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 -- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6018155f50 msgr2=0x7f60181563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6018155f50 0x7f60181563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 -- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000004d90 msgr2=0x7f6018150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000004d90 0x7f6018150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 -- 172.21.15.79:0/1184565797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180b4160 con 0x7f60180b3650 2023-06-08T03:50:19.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f60167fc700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 0x7f60180b4f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f600c00f840 tx=0x7f600c00fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c005320 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c0054a0 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60180b4440 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c006aa0 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.343+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60180b4a00 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.344+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60180d1290 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.345+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f600c006c00 con 0x7f60180b3650 2023-06-08T03:50:19.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.345+0000 7f5fff7fe700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f600406ac50 0x7f600406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.345+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f600c088520 con 0x7f60180b3650 2023-06-08T03:50:19.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.345+0000 7f6016ffd700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f600406ac50 0x7f600406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.346+0000 7f6016ffd700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f600406ac50 0x7f600406d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6008007d90 tx=0x7f6008000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:19.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.350+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f60180d1290 con 0x7f60180b3650 2023-06-08T03:50:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.548+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60180d1290 con 0x7f60180b3650 2023-06-08T03:50:19.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.549+0000 7f5fff7fe700 1 -- 172.21.15.79:0/1184565797 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14581 (secure 0 0 0) 0x7f60180d1290 con 0x7f60180b3650 2023-06-08T03:50:19.552 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:19.552 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:19.157125+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:19.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f600406ac50 msgr2=0x7f600406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f600406ac50 0x7f600406d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6008007d90 tx=0x7f6008000ed0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 msgr2=0x7f60180b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 0x7f60180b4f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f600c00f840 tx=0x7f600c00fb50 comp rx=0 tx=0).stop 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 shutdown_connections 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f600406ac50 0x7f600406d100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6018155f50 0x7f60181563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6000004d90 0x7f6018150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 --2- 172.21.15.79:0/1184565797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f60180b3650 0x7f60180b4f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 >> 172.21.15.79:0/1184565797 conn(0x7f601806a670 msgr2=0x7f60180cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:19.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 shutdown_connections 2023-06-08T03:50:19.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.551+0000 7f601d4d3700 1 -- 172.21.15.79:0/1184565797 wait complete. 2023-06-08T03:50:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-08T03:50:19.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:50:19.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.692+0000 7fafe9e39700 1 Processor -- start 2023-06-08T03:50:19.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.692+0000 7fafe9e39700 1 -- start start 2023-06-08T03:50:19.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe9e39700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe9e39700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fafe40cf490 0x7fafe40cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.698 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe9e39700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe40cae50 con 0x7fafe40cd080 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe9e39700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe40cbea0 con 0x7fafe40cf490 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe9e39700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40cda30 con 0x7fafe40d0060 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe2ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fafe40cd080 0x7fafe40b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57636/0 (socket says 172.21.15.79:57636) 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe2ffd700 1 -- 172.21.15.79:0/3459519659 learned_addr learned my addr 172.21.15.79:0/3459519659 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe3fff700 1 --2- 172.21.15.79:0/3459519659 >> v2:172.21.15.79:3300/0 conn(0x7fafe40cf490 0x7fafe40cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.693+0000 7fafe37fe700 1 --2- 172.21.15.79:0/3459519659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.699 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe27fc700 1 -- 172.21.15.79:0/3459519659 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580177688 0 0) 0x7fafe40cae50 con 0x7fafe40cd080 2023-06-08T03:50:19.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe27fc700 1 -- 172.21.15.79:0/3459519659 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd00035b0 con 0x7fafe40cd080 2023-06-08T03:50:19.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe3fff700 1 -- 172.21.15.79:0/3459519659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 msgr2=0x7fafe40af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:19.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe3fff700 1 --2- 172.21.15.79:0/3459519659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe3fff700 1 -- 172.21.15.79:0/3459519659 >> v1:172.21.15.79:6789/0 conn(0x7fafe40cd080 legacy=0x7fafe40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe3fff700 1 -- 172.21.15.79:0/3459519659 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe415d930 con 0x7fafe40cf490 2023-06-08T03:50:19.700 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe3fff700 1 --2- 172.21.15.79:0/3459519659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cf490 0x7fafe40cd4f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009840 tx=0x7fafdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da8d3ad371fbb33 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe27fc700 1 -- 172.21.15.79:0/3459519659 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc011830 con 0x7fafe40cf490 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe27fc700 1 -- 172.21.15.79:0/3459519659 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc011990 con 0x7fafe40cf490 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.694+0000 7fafe27fc700 1 -- 172.21.15.79:0/3459519659 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc011d20 con 0x7fafe40cf490 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 -- 172.21.15.79:0/3459519659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cf490 msgr2=0x7fafe40cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 --2- 172.21.15.79:0/3459519659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cf490 0x7fafe40cd4f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009840 tx=0x7fafdc009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 -- 172.21.15.79:0/3459519659 shutdown_connections 2023-06-08T03:50:19.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 --2- 172.21.15.79:0/3459519659 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cf490 0x7fafe40cd4f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 --2- 172.21.15.79:0/3459519659 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 -- 172.21.15.79:0/3459519659 >> 172.21.15.79:0/3459519659 conn(0x7fafe406e1e0 msgr2=0x7fafe406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:19.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 -- 172.21.15.79:0/3459519659 shutdown_connections 2023-06-08T03:50:19.702 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.695+0000 7fafe9e39700 1 -- 172.21.15.79:0/3459519659 wait complete. 2023-06-08T03:50:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.696+0000 7fafe9e39700 1 Processor -- start 2023-06-08T03:50:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.696+0000 7fafe9e39700 1 -- start start 2023-06-08T03:50:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe9e39700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 0x7fafe41f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.704 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe9e39700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe40cf490 0x7fafe40ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe9e39700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe9e39700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40c5eb0 con 0x7fafe40cd080 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe9e39700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40c6190 con 0x7fafe40cf490 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe9e39700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40c6470 con 0x7fafe40d0060 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 0x7fafe41f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 0x7fafe41f0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34164/0 (socket says 172.21.15.79:34164) 2023-06-08T03:50:19.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 -- 172.21.15.79:0/584067 learned_addr learned my addr 172.21.15.79:0/584067 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe3fff700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe2ffd700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe40cf490 0x7fafe40ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 -- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 msgr2=0x7fafe40ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 -- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe40cf490 msgr2=0x7fafe40ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe40cf490 0x7fafe40ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 -- 172.21.15.79:0/584067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0094a0 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.697+0000 7fafe37fe700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 0x7fafe41f0ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fafd400efc0 tx=0x7fafd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.698+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400cdc0 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.698+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400f040 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.698+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe40c6750 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.698+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40146a0 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.698+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe406cbc0 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.698+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe4073e90 con 0x7fafe40cd080 2023-06-08T03:50:19.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.699+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fafd4014800 con 0x7fafe40cd080 2023-06-08T03:50:19.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.699+0000 7fafdbfff700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fafc406ac50 0x7fafc406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:19.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.699+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fafd4089c40 con 0x7fafe40cd080 2023-06-08T03:50:19.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.700+0000 7fafe2ffd700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fafc406ac50 0x7fafc406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:19.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.700+0000 7fafe2ffd700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fafc406ac50 0x7fafc406d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fafe41f4c70 tx=0x7fafcc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:19.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.704+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fafd4090080 con 0x7fafe40cd080 2023-06-08T03:50:19.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.898+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fafe40c7720 con 0x7fafe40cd080 2023-06-08T03:50:19.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.898+0000 7fafdbfff700 1 -- 172.21.15.79:0/584067 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fafd40551b0 con 0x7fafe40cd080 2023-06-08T03:50:19.902 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:50:19.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fafc406ac50 msgr2=0x7fafc406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fafc406ac50 0x7fafc406d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fafe41f4c70 tx=0x7fafcc009040 comp rx=0 tx=0).stop 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 msgr2=0x7fafe41f0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 0x7fafe41f0ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fafd400efc0 tx=0x7fafd400c3a0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 shutdown_connections 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fafc406ac50 0x7fafc406d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fafe40d0060 0x7fafe40ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fafe40cd080 0x7fafe41f0ef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 --2- 172.21.15.79:0/584067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe40cf490 0x7fafe40ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:19.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 >> 172.21.15.79:0/584067 conn(0x7fafe406e1e0 msgr2=0x7fafe40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:19.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 shutdown_connections 2023-06-08T03:50:19.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:19.900+0000 7fafe9e39700 1 -- 172.21.15.79:0/584067 wait complete. 2023-06-08T03:50:19.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:50:20.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.034+0000 7f5fa377e700 1 Processor -- start 2023-06-08T03:50:20.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.034+0000 7f5fa377e700 1 -- start start 2023-06-08T03:50:20.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa377e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0d0060 0x7f5f9c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa377e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5f9c0cf490 0x7f5f9c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa377e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c0cae50 con 0x7f5f9c0cd080 2023-06-08T03:50:20.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa377e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0cbea0 con 0x7f5f9c0cf490 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa377e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0cda30 con 0x7f5f9c0d0060 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5f9c0cf490 0x7f5f9c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5f9c0cf490 0x7f5f9c0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34176/0 (socket says 172.21.15.79:34176) 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 -- 172.21.15.79:0/2402566801 learned_addr learned my addr 172.21.15.79:0/2402566801 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa151a700 1 --2- 172.21.15.79:0/2402566801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0d0060 0x7f5f9c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 -- 172.21.15.79:0/2402566801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0d0060 msgr2=0x7f5f9c0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:20.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 --2- 172.21.15.79:0/2402566801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0d0060 0x7f5f9c0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 -- 172.21.15.79:0/2402566801 >> v1:172.21.15.156:6789/0 conn(0x7f5f9c0cd080 legacy=0x7f5f9c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 -- 172.21.15.79:0/2402566801 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c15d930 con 0x7f5f9c0cf490 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5fa1d1b700 1 --2- 172.21.15.79:0/2402566801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0cd4f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009e50 tx=0x7f5f9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9da2879dbc9f3b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5f93fff700 1 -- 172.21.15.79:0/2402566801 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400e070 con 0x7f5f9c0cf490 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5f93fff700 1 -- 172.21.15.79:0/2402566801 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9400b910 con 0x7f5f9c0cf490 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.035+0000 7f5f93fff700 1 -- 172.21.15.79:0/2402566801 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400bcc0 con 0x7f5f9c0cf490 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 -- 172.21.15.79:0/2402566801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 msgr2=0x7f5f9c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 --2- 172.21.15.79:0/2402566801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0cd4f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009e50 tx=0x7f5f9400b330 comp rx=0 tx=0).stop 2023-06-08T03:50:20.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 -- 172.21.15.79:0/2402566801 shutdown_connections 2023-06-08T03:50:20.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 --2- 172.21.15.79:0/2402566801 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0cd4f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 --2- 172.21.15.79:0/2402566801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0d0060 0x7f5f9c0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 -- 172.21.15.79:0/2402566801 >> 172.21.15.79:0/2402566801 conn(0x7f5f9c06e1e0 msgr2=0x7f5f9c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:20.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 -- 172.21.15.79:0/2402566801 shutdown_connections 2023-06-08T03:50:20.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.036+0000 7f5fa377e700 1 -- 172.21.15.79:0/2402566801 wait complete. 2023-06-08T03:50:20.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.037+0000 7f5fa377e700 1 Processor -- start 2023-06-08T03:50:20.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.037+0000 7f5fa377e700 1 -- start start 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa377e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cd080 0x7f5f9c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa377e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa377e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0d0060 0x7f5f9c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa377e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c15af30 con 0x7f5f9c0cf490 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa377e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c85a0 con 0x7f5f9c0d0060 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa377e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c8860 con 0x7f5f9c0cd080 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa0d19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa0d19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34180/0 (socket says 172.21.15.79:34180) 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa0d19700 1 -- 172.21.15.79:0/3072223282 learned_addr learned my addr 172.21.15.79:0/3072223282 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa151a700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cd080 0x7f5f9c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa1d1b700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0d0060 0x7f5f9c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.038+0000 7f5fa0d19700 1 -- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cd080 msgr2=0x7f5f9c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa0d19700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cd080 0x7f5f9c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa0d19700 1 -- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0d0060 msgr2=0x7f5f9c150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.046 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa0d19700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0d0060 0x7f5f9c150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa0d19700 1 -- 172.21.15.79:0/3072223282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94009b30 con 0x7f5f9c0cf490 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa0d19700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0c4180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00c420 tx=0x7f5f8c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0052d0 con 0x7f5f9c0cf490 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c005430 con 0x7f5f9c0cf490 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c0c8b40 con 0x7f5f9c0cf490 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0038c0 con 0x7f5f9c0cf490 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.039+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c0c9100 con 0x7f5f9c0cf490 2023-06-08T03:50:20.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.040+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c073e90 con 0x7f5f9c0cf490 2023-06-08T03:50:20.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.040+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f5f8c003a20 con 0x7f5f9c0cf490 2023-06-08T03:50:20.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.041+0000 7f5f91ffb700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5f8006ac50 0x7f5f8006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.041+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5f8c088330 con 0x7f5f9c0cf490 2023-06-08T03:50:20.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.041+0000 7f5fa151a700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5f8006ac50 0x7f5f8006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.041+0000 7f5fa151a700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5f8006ac50 0x7f5f8006d100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009be0 tx=0x7f5f98009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.048 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.045+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f8c053810 con 0x7f5f9c0cf490 2023-06-08T03:50:20.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.238+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f9c0c3f20 con 0x7f5f9c0cf490 2023-06-08T03:50:20.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.238+0000 7f5f91ffb700 1 -- 172.21.15.79:0/3072223282 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f8c056e30 con 0x7f5f9c0cf490 2023-06-08T03:50:20.242 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:50:20.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5f8006ac50 msgr2=0x7f5f8006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5f8006ac50 0x7f5f8006d100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009be0 tx=0x7f5f98009210 comp rx=0 tx=0).stop 2023-06-08T03:50:20.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 msgr2=0x7f5f9c0c4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0c4180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00c420 tx=0x7f5f8c00c730 comp rx=0 tx=0).stop 2023-06-08T03:50:20.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 shutdown_connections 2023-06-08T03:50:20.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f9c0d0060 0x7f5f9c150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5f8006ac50 0x7f5f8006d100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f9c0cd080 0x7f5f9c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 --2- 172.21.15.79:0/3072223282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5f9c0cf490 0x7f5f9c0c4180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 >> 172.21.15.79:0/3072223282 conn(0x7f5f9c06e1e0 msgr2=0x7f5f9c0b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:20.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 shutdown_connections 2023-06-08T03:50:20.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.241+0000 7f5fa377e700 1 -- 172.21.15.79:0/3072223282 wait complete. 2023-06-08T03:50:20.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:20.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.377+0000 7f75e2cf7700 1 Processor -- start 2023-06-08T03:50:20.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 -- start start 2023-06-08T03:50:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f75dc158e30 0x7f75dc0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc159200 0x7f75dc0cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75dc15d730 0x7f75dc0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc0caef0 con 0x7f75dc15d730 2023-06-08T03:50:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc0cb1d0 con 0x7f75dc158e30 2023-06-08T03:50:20.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e2cf7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0cb4b0 con 0x7f75dc159200 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e1294700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75dc15d730 0x7f75dc0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.378+0000 7f75e1294700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75dc15d730 0x7f75dc0cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34194/0 (socket says 172.21.15.79:34194) 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 -- 172.21.15.79:0/117757968 learned_addr learned my addr 172.21.15.79:0/117757968 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e0a93700 1 --2- 172.21.15.79:0/117757968 >> v2:172.21.15.156:3300/0 conn(0x7f75dc158e30 0x7f75dc0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75dbfff700 1 --2- 172.21.15.79:0/117757968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc159200 0x7f75dc0cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 -- 172.21.15.79:0/117757968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc159200 msgr2=0x7f75dc0cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 --2- 172.21.15.79:0/117757968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc159200 0x7f75dc0cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 -- 172.21.15.79:0/117757968 >> v2:172.21.15.156:3300/0 conn(0x7f75dc158e30 msgr2=0x7f75dc0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 --2- 172.21.15.79:0/117757968 >> v2:172.21.15.156:3300/0 conn(0x7f75dc158e30 0x7f75dc0ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 -- 172.21.15.79:0/117757968 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0cb790 con 0x7f75dc15d730 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75e1294700 1 --2- 172.21.15.79:0/117757968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc15d730 0x7f75dc0cc2c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75dc06cbe0 tx=0x7f75d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d8fa1f84b93146 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75db7fe700 1 -- 172.21.15.79:0/117757968 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400e070 con 0x7f75dc15d730 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75db7fe700 1 -- 172.21.15.79:0/117757968 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d400b9c0 con 0x7f75dc15d730 2023-06-08T03:50:20.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.379+0000 7f75db7fe700 1 -- 172.21.15.79:0/117757968 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d400bd70 con 0x7f75dc15d730 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 -- 172.21.15.79:0/117757968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc15d730 msgr2=0x7f75dc0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/117757968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc15d730 0x7f75dc0cc2c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75dc06cbe0 tx=0x7f75d400b330 comp rx=0 tx=0).stop 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 -- 172.21.15.79:0/117757968 shutdown_connections 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/117757968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc15d730 0x7f75dc0cc2c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/117757968 >> v2:172.21.15.156:3300/0 conn(0x7f75dc158e30 0x7f75dc0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/117757968 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc159200 0x7f75dc0cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 -- 172.21.15.79:0/117757968 >> 172.21.15.79:0/117757968 conn(0x7f75dc0b1d20 msgr2=0x7f75dc06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:20.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 -- 172.21.15.79:0/117757968 shutdown_connections 2023-06-08T03:50:20.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.380+0000 7f75e2cf7700 1 -- 172.21.15.79:0/117757968 wait complete. 2023-06-08T03:50:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.381+0000 7f75e2cf7700 1 Processor -- start 2023-06-08T03:50:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.381+0000 7f75e2cf7700 1 -- start start 2023-06-08T03:50:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 0x7f75dc0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75dc159200 0x7f75dc15bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc15d730 0x7f75dc1f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0acff0 con 0x7f75dc158e30 2023-06-08T03:50:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0ac450 con 0x7f75dc159200 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc0ac730 con 0x7f75dc15d730 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 0x7f75dc0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 0x7f75dc0c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34196/0 (socket says 172.21.15.79:34196) 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 -- 172.21.15.79:0/3283079925 learned_addr learned my addr 172.21.15.79:0/3283079925 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75dbfff700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75dc159200 0x7f75dc15bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e1294700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc15d730 0x7f75dc1f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 -- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc15d730 msgr2=0x7f75dc1f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc15d730 0x7f75dc1f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 -- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75dc159200 msgr2=0x7f75dc15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75dc159200 0x7f75dc15bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 -- 172.21.15.79:0/3283079925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4009b30 con 0x7f75dc158e30 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e0a93700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 0x7f75dc0c6640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d800 tx=0x7f75d000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00125f0 con 0x7f75dc158e30 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0012750 con 0x7f75dc158e30 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc0aca10 con 0x7f75dc158e30 2023-06-08T03:50:20.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d0012b00 con 0x7f75dc158e30 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.382+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc1f1e80 con 0x7f75dc158e30 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.383+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc0acc00 con 0x7f75dc158e30 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.384+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f75d0012c60 con 0x7f75dc158e30 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.384+0000 7f75d97fa700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75c006ac50 0x7f75c006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.384+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f75d0088ff0 con 0x7f75dc158e30 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.384+0000 7f75dbfff700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75c006ac50 0x7f75c006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.385+0000 7f75dbfff700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75c006ac50 0x7f75c006d100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75dc157ea0 tx=0x7f75c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.389+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75d00bb310 con 0x7f75dc158e30 2023-06-08T03:50:20.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.616+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75dc073e90 con 0x7f75dc158e30 2023-06-08T03:50:20.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.617+0000 7f75d97fa700 1 -- 172.21.15.79:0/3283079925 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f75d00544d0 con 0x7f75dc158e30 2023-06-08T03:50:20.620 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:20.621 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-08T03:50:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75c006ac50 msgr2=0x7f75c006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75c006ac50 0x7f75c006d100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75dc157ea0 tx=0x7f75c8008040 comp rx=0 tx=0).stop 2023-06-08T03:50:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 msgr2=0x7f75dc0c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 0x7f75dc0c6640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75d000d800 tx=0x7f75d000db10 comp rx=0 tx=0).stop 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 shutdown_connections 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f75c006ac50 0x7f75c006d100 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f75dc15d730 0x7f75dc1f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75dc158e30 0x7f75dc0c6640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 --2- 172.21.15.79:0/3283079925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75dc159200 0x7f75dc15bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 >> 172.21.15.79:0/3283079925 conn(0x7f75dc0b1d20 msgr2=0x7f75dc0ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:20.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 shutdown_connections 2023-06-08T03:50:20.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.619+0000 7f75e2cf7700 1 -- 172.21.15.79:0/3283079925 wait complete. 2023-06-08T03:50:20.624 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 3 2023-06-08T03:50:20.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.749+0000 7fc9eef1e700 1 Processor -- start 2023-06-08T03:50:20.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9eef1e700 1 -- start start 2023-06-08T03:50:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9eef1e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9e8158e30 0x7fc9e80ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9eef1e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e80caef0 con 0x7fc9e8159200 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9eef1e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e80cb1d0 con 0x7fc9e80cc490 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9eef1e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e80cb4b0 con 0x7fc9e8158e30 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9e7fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc9e8159200 0x7fc9e80cbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57680/0 (socket says 172.21.15.79:57680) 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.750+0000 7fc9e7fff700 1 -- 172.21.15.79:0/317398625 learned_addr learned my addr 172.21.15.79:0/317398625 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 --2- 172.21.15.79:0/317398625 >> v2:172.21.15.156:3300/0 conn(0x7fc9e8158e30 0x7fc9e80ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614096276 0 0) 0x7fc9e80caef0 con 0x7fc9e8159200 2023-06-08T03:50:20.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d80035b0 con 0x7fc9e8159200 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615824304 0 0) 0x7fc9e80cb1d0 con 0x7fc9e80cc490 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e80caef0 con 0x7fc9e80cc490 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065486012 0 0) 0x7fc9d80035b0 con 0x7fc9e8159200 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 >> v2:172.21.15.156:3300/0 conn(0x7fc9e8158e30 msgr2=0x7fc9e80ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 --2- 172.21.15.79:0/317398625 >> v2:172.21.15.156:3300/0 conn(0x7fc9e8158e30 0x7fc9e80ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 >> v1:172.21.15.156:6789/0 conn(0x7fc9e80cc490 legacy=0x7fc9e8071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80cb790 con 0x7fc9e8159200 2023-06-08T03:50:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fc9d4001670 con 0x7fc9e8159200 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 >> v1:172.21.15.79:6789/0 conn(0x7fc9e8159200 legacy=0x7fc9e80cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9d8003c00 0x7fc9d8003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d8004850 0x7fc9d8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8005200 0x7fc9d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80cb4b0 con 0x7fc9d8003c00 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d800c0e0 con 0x7fc9d8004850 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d800c2a0 con 0x7fc9d8005200 2023-06-08T03:50:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 --2- 172.21.15.79:0/317398625 >> v2:172.21.15.156:3300/0 conn(0x7fc9e8158e30 0x7fc9e80ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e7fff700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d8004850 0x7fc9d8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9d8003c00 0x7fc9d8003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9ed4bb700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8005200 0x7fc9d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 -- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8005200 msgr2=0x7fc9d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8005200 0x7fc9d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 -- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d8004850 msgr2=0x7fc9d8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d8004850 0x7fc9d8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9eccba700 1 -- 172.21.15.79:0/317398625 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80035b0 con 0x7fc9d8003c00 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9ed4bb700 1 -- 172.21.15.79:0/317398625 reap_dead start 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.751+0000 7fc9e7fff700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9d8004850 0x7fc9d8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eccba700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9d8003c00 0x7fc9d8003ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009ce0 tx=0x7fc9d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a79b5b3ef5d06f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc004030 con 0x7fc9d8003c00 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc004190 con 0x7fc9d8003c00 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9e77fe700 1 -- 172.21.15.79:0/317398625 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc004520 con 0x7fc9d8003c00 2023-06-08T03:50:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 -- 172.21.15.79:0/317398625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9d8003c00 msgr2=0x7fc9d8003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9d8003c00 0x7fc9d8003ff0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009ce0 tx=0x7fc9d8005880 comp rx=0 tx=0).stop 2023-06-08T03:50:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 -- 172.21.15.79:0/317398625 shutdown_connections 2023-06-08T03:50:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/317398625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9d8003c00 0x7fc9d8003ff0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 -- 172.21.15.79:0/317398625 >> 172.21.15.79:0/317398625 conn(0x7fc9e80b1d20 msgr2=0x7fc9e806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 -- 172.21.15.79:0/317398625 shutdown_connections 2023-06-08T03:50:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.752+0000 7fc9eef1e700 1 -- 172.21.15.79:0/317398625 wait complete. 2023-06-08T03:50:20.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.753+0000 7fc9eef1e700 1 Processor -- start 2023-06-08T03:50:20.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 -- start start 2023-06-08T03:50:20.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8003c00 0x7fc9e80c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 0x7fc9e814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e80b3500 0x7fc9e80b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80c5ee0 con 0x7fc9e814b2e0 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80b4260 con 0x7fc9e80b3500 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.754+0000 7fc9eef1e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80b4500 con 0x7fc9d8003c00 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 0x7fc9e814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 0x7fc9e814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34218/0 (socket says 172.21.15.79:34218) 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 -- 172.21.15.79:0/1184938983 learned_addr learned my addr 172.21.15.79:0/1184938983 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9ed4bb700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e80b3500 0x7fc9e80b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9eccba700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8003c00 0x7fc9e80c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 -- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8003c00 msgr2=0x7fc9e80c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8003c00 0x7fc9e80c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 -- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e80b3500 msgr2=0x7fc9e80b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e80b3500 0x7fc9e80b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 -- 172.21.15.79:0/1184938983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80b47e0 con 0x7fc9e814b2e0 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9ed4bb700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e80b3500 0x7fc9e80b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e7fff700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 0x7fc9e814aa30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4000fb0 tx=0x7fc9d4007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400f5e0 con 0x7fc9e814b2e0 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400f740 con 0x7fc9e814b2e0 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e80c6640 con 0x7fc9e814b2e0 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400fab0 con 0x7fc9e814b2e0 2023-06-08T03:50:20.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.755+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e80b3eb0 con 0x7fc9e814b2e0 2023-06-08T03:50:20.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.756+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e80b40d0 con 0x7fc9e814b2e0 2023-06-08T03:50:20.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.757+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fc9d400fc10 con 0x7fc9e814b2e0 2023-06-08T03:50:20.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.757+0000 7fc9e57fa700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc9cc06ac50 0x7fc9cc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:20.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.757+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9d4089970 con 0x7fc9e814b2e0 2023-06-08T03:50:20.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.758+0000 7fc9eccba700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc9cc06ac50 0x7fc9cc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:20.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.758+0000 7fc9eccba700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc9cc06ac50 0x7fc9cc06d100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0090b0 tx=0x7fc9dc01d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:20.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.761+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc9d4054e40 con 0x7fc9e814b2e0 2023-06-08T03:50:20.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.881+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9d4055210 con 0x7fc9e814b2e0 2023-06-08T03:50:20.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.954+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9e80c6830 con 0x7fc9e814b2e0 2023-06-08T03:50:20.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.954+0000 7fc9e57fa700 1 -- 172.21.15.79:0/1184938983 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14555 (secure 0 0 0) 0x7fc9d4043620 con 0x7fc9e814b2e0 2023-06-08T03:50:20.957 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:20.958 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:20.167431+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:20.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.956+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc9cc06ac50 msgr2=0x7fc9cc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.956+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc9cc06ac50 0x7fc9cc06d100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0090b0 tx=0x7fc9dc01d040 comp rx=0 tx=0).stop 2023-06-08T03:50:20.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.956+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 msgr2=0x7fc9e814aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:20.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.956+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 0x7fc9e814aa30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4000fb0 tx=0x7fc9d4007570 comp rx=0 tx=0).stop 2023-06-08T03:50:20.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 shutdown_connections 2023-06-08T03:50:20.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9e80b3500 0x7fc9e80b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc9cc06ac50 0x7fc9cc06d100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9d8003c00 0x7fc9e80c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 --2- 172.21.15.79:0/1184938983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9e814b2e0 0x7fc9e814aa30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:20.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 >> 172.21.15.79:0/1184938983 conn(0x7fc9e80b1d20 msgr2=0x7fc9e8071710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:20.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 shutdown_connections 2023-06-08T03:50:20.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:20.957+0000 7fc9eef1e700 1 -- 172.21.15.79:0/1184938983 wait complete. 2023-06-08T03:50:20.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-08T03:50:20.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:50:21.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.090+0000 7f303ed74700 1 Processor -- start 2023-06-08T03:50:21.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 -- start start 2023-06-08T03:50:21.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f303815c1f0 0x7f303806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3038071fe0 0x7f30380cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30380cf130 0x7f3038069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30380cd140 con 0x7f3038071fe0 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3038071110 con 0x7f30380cf130 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.091+0000 7f303ed74700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3038071be0 con 0x7f303815c1f0 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3038071fe0 0x7f30380cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3038071fe0 0x7f30380cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34224/0 (socket says 172.21.15.79:34224) 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 -- 172.21.15.79:0/1669437692 learned_addr learned my addr 172.21.15.79:0/1669437692 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:21.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f303d311700 1 --2- 172.21.15.79:0/1669437692 >> v2:172.21.15.156:3300/0 conn(0x7f30380cf130 0x7f3038069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 -- 172.21.15.79:0/1669437692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f303815c1f0 msgr2=0x7f303806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 --2- 172.21.15.79:0/1669437692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f303815c1f0 0x7f303806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 -- 172.21.15.79:0/1669437692 >> v2:172.21.15.156:3300/0 conn(0x7f30380cf130 msgr2=0x7f3038069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 --2- 172.21.15.79:0/1669437692 >> v2:172.21.15.156:3300/0 conn(0x7f30380cf130 0x7f3038069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 -- 172.21.15.79:0/1669437692 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380cd390 con 0x7f3038071fe0 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f303d311700 1 --2- 172.21.15.79:0/1669437692 >> v2:172.21.15.156:3300/0 conn(0x7f30380cf130 0x7f3038069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f3037fff700 1 --2- 172.21.15.79:0/1669437692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3038071fe0 0x7f30380cebf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3038070e40 tx=0x7f3024009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba000b7dca0c85d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f30377fe700 1 -- 172.21.15.79:0/1669437692 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302400b870 con 0x7f3038071fe0 2023-06-08T03:50:21.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f30377fe700 1 -- 172.21.15.79:0/1669437692 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302400b9d0 con 0x7f3038071fe0 2023-06-08T03:50:21.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.092+0000 7f30377fe700 1 -- 172.21.15.79:0/1669437692 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302400bd60 con 0x7f3038071fe0 2023-06-08T03:50:21.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 -- 172.21.15.79:0/1669437692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3038071fe0 msgr2=0x7f30380cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 --2- 172.21.15.79:0/1669437692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3038071fe0 0x7f30380cebf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3038070e40 tx=0x7f3024009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 -- 172.21.15.79:0/1669437692 shutdown_connections 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 --2- 172.21.15.79:0/1669437692 >> v2:172.21.15.156:3300/0 conn(0x7f30380cf130 0x7f3038069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 --2- 172.21.15.79:0/1669437692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f303815c1f0 0x7f303806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 --2- 172.21.15.79:0/1669437692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3038071fe0 0x7f30380cebf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 -- 172.21.15.79:0/1669437692 >> 172.21.15.79:0/1669437692 conn(0x7f30380b1d30 msgr2=0x7f303806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 -- 172.21.15.79:0/1669437692 shutdown_connections 2023-06-08T03:50:21.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.093+0000 7f303ed74700 1 -- 172.21.15.79:0/1669437692 wait complete. 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.094+0000 7f303ed74700 1 Processor -- start 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.094+0000 7f303ed74700 1 -- start start 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303ed74700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3038071fe0 0x7f3038151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303ed74700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 0x7f30380c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303ed74700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f303815c1f0 0x7f3038151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303ed74700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30380b4e50 con 0x7f30380cf130 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303ed74700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30381518b0 con 0x7f303815c1f0 2023-06-08T03:50:21.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303ed74700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3038151a70 con 0x7f3038071fe0 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 0x7f30380c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 0x7f30380c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34230/0 (socket says 172.21.15.79:34230) 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 -- 172.21.15.79:0/1367908439 learned_addr learned my addr 172.21.15.79:0/1367908439 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303cb10700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3038071fe0 0x7f3038151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303d311700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f303815c1f0 0x7f3038151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 -- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3038071fe0 msgr2=0x7f3038151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3038071fe0 0x7f3038151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 -- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f303815c1f0 msgr2=0x7f3038151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f303815c1f0 0x7f3038151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 -- 172.21.15.79:0/1367908439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009550 con 0x7f30380cf130 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303d311700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f303815c1f0 0x7f3038151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f303cb10700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3038071fe0 0x7f3038151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f3037fff700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 0x7f30380c9190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3038070770 tx=0x7f3024004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.095+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024004520 con 0x7f30380cf130 2023-06-08T03:50:21.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.096+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302401e070 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.096+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30380c8a20 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.096+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240038e0 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.096+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30380c8d00 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.096+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30380c8e90 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.097+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3024003a40 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.097+0000 7f30357fa700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f301c06ace0 0x7f301c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.097+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3024020070 con 0x7f30380cf130 2023-06-08T03:50:21.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.098+0000 7f303cb10700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f301c06ace0 0x7f301c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.098+0000 7f303cb10700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f301c06ace0 0x7f301c06d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f302c006fd0 tx=0x7f302c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.102+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3024055c20 con 0x7f30380cf130 2023-06-08T03:50:21.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.287+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3038073e90 con 0x7f30380cf130 2023-06-08T03:50:21.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.288+0000 7f30357fa700 1 -- 172.21.15.79:0/1367908439 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3024059240 con 0x7f30380cf130 2023-06-08T03:50:21.291 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:50:21.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f301c06ace0 msgr2=0x7f301c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f301c06ace0 0x7f301c06d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f302c006fd0 tx=0x7f302c008040 comp rx=0 tx=0).stop 2023-06-08T03:50:21.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 msgr2=0x7f30380c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 0x7f30380c9190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3038070770 tx=0x7f3024004750 comp rx=0 tx=0).stop 2023-06-08T03:50:21.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 shutdown_connections 2023-06-08T03:50:21.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f303815c1f0 0x7f3038151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f301c06ace0 0x7f301c06d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3038071fe0 0x7f3038151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 --2- 172.21.15.79:0/1367908439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30380cf130 0x7f30380c9190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 >> 172.21.15.79:0/1367908439 conn(0x7f30380b1d30 msgr2=0x7f303815e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:21.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 shutdown_connections 2023-06-08T03:50:21.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.290+0000 7f303ed74700 1 -- 172.21.15.79:0/1367908439 wait complete. 2023-06-08T03:50:21.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:50:21.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.423+0000 7f76769ab700 1 Processor -- start 2023-06-08T03:50:21.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.423+0000 7f76769ab700 1 -- start start 2023-06-08T03:50:21.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f76769ab700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700d0060 0x7f76700af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f76769ab700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f76700cd080 0x7f76700cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f76769ab700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76700cae50 con 0x7f76700b20f0 2023-06-08T03:50:21.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f76769ab700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76700cbea0 con 0x7f76700cd080 2023-06-08T03:50:21.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f76769ab700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76700cda30 con 0x7f76700d0060 2023-06-08T03:50:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f766f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f76700cd080 0x7f76700cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f7674f48700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f76700b20f0 0x7f76700b2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57712/0 (socket says 172.21.15.79:57712) 2023-06-08T03:50:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.424+0000 7f7674f48700 1 -- 172.21.15.79:0/2708876718 learned_addr learned my addr 172.21.15.79:0/2708876718 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766ffff700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700d0060 0x7f76700af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766f7fe700 1 -- 172.21.15.79:0/2708876718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700d0060 msgr2=0x7f76700af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766f7fe700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700d0060 0x7f76700af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766f7fe700 1 -- 172.21.15.79:0/2708876718 >> v1:172.21.15.79:6789/0 conn(0x7f76700b20f0 legacy=0x7f76700b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766effd700 1 -- 172.21.15.79:0/2708876718 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867646042 0 0) 0x7f76700cae50 con 0x7f76700b20f0 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766f7fe700 1 -- 172.21.15.79:0/2708876718 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767015d930 con 0x7f76700cd080 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766ffff700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700d0060 0x7f76700af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766f7fe700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700cd4f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7660009840 tx=0x7f7660009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3745eb48c1d16a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766effd700 1 -- 172.21.15.79:0/2708876718 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660011830 con 0x7f76700cd080 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766effd700 1 -- 172.21.15.79:0/2708876718 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660011990 con 0x7f76700cd080 2023-06-08T03:50:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f766effd700 1 -- 172.21.15.79:0/2708876718 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660011d20 con 0x7f76700cd080 2023-06-08T03:50:21.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f76769ab700 1 -- 172.21.15.79:0/2708876718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 msgr2=0x7f76700cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.425+0000 7f76769ab700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700cd4f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7660009840 tx=0x7f7660009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:21.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.426+0000 7f76769ab700 1 -- 172.21.15.79:0/2708876718 shutdown_connections 2023-06-08T03:50:21.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.426+0000 7f76769ab700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700d0060 0x7f76700af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.426+0000 7f76769ab700 1 --2- 172.21.15.79:0/2708876718 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700cd4f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.426+0000 7f76769ab700 1 -- 172.21.15.79:0/2708876718 >> 172.21.15.79:0/2708876718 conn(0x7f767006e1e0 msgr2=0x7f767006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.426+0000 7f76769ab700 1 -- 172.21.15.79:0/2708876718 shutdown_connections 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.426+0000 7f76769ab700 1 -- 172.21.15.79:0/2708876718 wait complete. 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.427+0000 7f76769ab700 1 Processor -- start 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.427+0000 7f76769ab700 1 -- start start 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f76769ab700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700b20f0 0x7f76700a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f76769ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f76769ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76700d0060 0x7f76700aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f76769ab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7670150670 con 0x7f76700cd080 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f76769ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700a9250 con 0x7f76700d0060 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f76769ab700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76700a85d0 con 0x7f76700b20f0 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f766f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f766f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700a7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34244/0 (socket says 172.21.15.79:34244) 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f766f7fe700 1 -- 172.21.15.79:0/2055000838 learned_addr learned my addr 172.21.15.79:0/2055000838 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:21.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.428+0000 7f766ffff700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700b20f0 0x7f76700a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766f7fe700 1 -- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700b20f0 msgr2=0x7f76700a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f7674f48700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76700d0060 0x7f76700aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766f7fe700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700b20f0 0x7f76700a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766f7fe700 1 -- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76700d0060 msgr2=0x7f76700aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766f7fe700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76700d0060 0x7f76700aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766f7fe700 1 -- 172.21.15.79:0/2055000838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600094a0 con 0x7f76700cd080 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766f7fe700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700a7b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f7660004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600045f0 con 0x7f76700cd080 2023-06-08T03:50:21.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766001f070 con 0x7f76700cd080 2023-06-08T03:50:21.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76700a88b0 con 0x7f76700cd080 2023-06-08T03:50:21.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660003ae0 con 0x7f76700cd080 2023-06-08T03:50:21.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.429+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76700a8e70 con 0x7f76700cd080 2023-06-08T03:50:21.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.430+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76700a9390 con 0x7f76700cd080 2023-06-08T03:50:21.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.430+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7660003c40 con 0x7f76700cd080 2023-06-08T03:50:21.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.431+0000 7f766cff9700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f765c06ace0 0x7f765c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.431+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7660021070 con 0x7f76700cd080 2023-06-08T03:50:21.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.431+0000 7f766ffff700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f765c06ace0 0x7f765c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.431+0000 7f766ffff700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f765c06ace0 0x7f765c06d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7664006fd0 tx=0x7f7664008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.435+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f76700a9390 con 0x7f76700cd080 2023-06-08T03:50:21.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.622+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76700a9390 con 0x7f76700cd080 2023-06-08T03:50:21.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.622+0000 7f766cff9700 1 -- 172.21.15.79:0/2055000838 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76700a9390 con 0x7f76700cd080 2023-06-08T03:50:21.625 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:50:21.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f765c06ace0 msgr2=0x7f765c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f765c06ace0 0x7f765c06d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7664006fd0 tx=0x7f7664008040 comp rx=0 tx=0).stop 2023-06-08T03:50:21.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 msgr2=0x7f76700a7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700a7b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c00 tx=0x7f7660004750 comp rx=0 tx=0).stop 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 shutdown_connections 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76700d0060 0x7f76700aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f765c06ace0 0x7f765c06d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76700b20f0 0x7f76700a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 --2- 172.21.15.79:0/2055000838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76700cd080 0x7f76700a7b10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 >> 172.21.15.79:0/2055000838 conn(0x7f767006e1e0 msgr2=0x7f76700b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:21.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 shutdown_connections 2023-06-08T03:50:21.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.624+0000 7f76769ab700 1 -- 172.21.15.79:0/2055000838 wait complete. 2023-06-08T03:50:21.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-08T03:50:21.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.754+0000 7fb197f0b700 1 Processor -- start 2023-06-08T03:50:21.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.755+0000 7fb197f0b700 1 -- start start 2023-06-08T03:50:21.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.755+0000 7fb197f0b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb19006b0d0 0x7fb1900ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.755+0000 7fb197f0b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1900cea60 con 0x7fb1900b1650 2023-06-08T03:50:21.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.755+0000 7fb197f0b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1900ced40 con 0x7fb190160080 2023-06-08T03:50:21.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.755+0000 7fb197f0b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1900cf6e0 con 0x7fb19006b0d0 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb195ca7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb19006b0d0 0x7fb1900ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb1954a6700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb1900b1650 0x7fb19015f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57726/0 (socket says 172.21.15.79:57726) 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb1954a6700 1 -- 172.21.15.79:0/957572435 learned_addr learned my addr 172.21.15.79:0/957572435 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb195ca7700 1 -- 172.21.15.79:0/957572435 >> v1:172.21.15.156:6789/0 conn(0x7fb190160080 legacy=0x7fb1900cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb195ca7700 1 -- 172.21.15.79:0/957572435 >> v1:172.21.15.79:6789/0 conn(0x7fb1900b1650 legacy=0x7fb19015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb195ca7700 1 -- 172.21.15.79:0/957572435 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1900cf020 con 0x7fb19006b0d0 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb194ca5700 1 -- 172.21.15.79:0/957572435 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852468009 0 0) 0x7fb1900cea60 con 0x7fb1900b1650 2023-06-08T03:50:21.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb195ca7700 1 --2- 172.21.15.79:0/957572435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb19006b0d0 0x7fb1900ce460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb180009840 tx=0x7fb180009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a483004a970dd78 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb194ca5700 1 -- 172.21.15.79:0/957572435 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180011830 con 0x7fb19006b0d0 2023-06-08T03:50:21.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb194ca5700 1 -- 172.21.15.79:0/957572435 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb180011990 con 0x7fb19006b0d0 2023-06-08T03:50:21.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.756+0000 7fb194ca5700 1 -- 172.21.15.79:0/957572435 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180011d20 con 0x7fb19006b0d0 2023-06-08T03:50:21.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 -- 172.21.15.79:0/957572435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb19006b0d0 msgr2=0x7fb1900ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 --2- 172.21.15.79:0/957572435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb19006b0d0 0x7fb1900ce460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb180009840 tx=0x7fb180009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:21.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 -- 172.21.15.79:0/957572435 shutdown_connections 2023-06-08T03:50:21.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 --2- 172.21.15.79:0/957572435 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb19006b0d0 0x7fb1900ce460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 -- 172.21.15.79:0/957572435 >> 172.21.15.79:0/957572435 conn(0x7fb19006a670 msgr2=0x7fb190099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 -- 172.21.15.79:0/957572435 shutdown_connections 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.757+0000 7fb197f0b700 1 -- 172.21.15.79:0/957572435 wait complete. 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.758+0000 7fb197f0b700 1 Processor -- start 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.758+0000 7fb197f0b700 1 -- start start 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb19006b0d0 0x7fb1900c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1900b1650 0x7fb19015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 0x7fb19009af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900ad350 con 0x7fb190160080 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900c7130 con 0x7fb19006b0d0 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1900ad560 con 0x7fb1900b1650 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 0x7fb19009af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 0x7fb19009af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34260/0 (socket says 172.21.15.79:34260) 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 -- 172.21.15.79:0/4164352370 learned_addr learned my addr 172.21.15.79:0/4164352370 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:21.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1954a6700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1900b1650 0x7fb19015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb195ca7700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb19006b0d0 0x7fb1900c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 -- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1900b1650 msgr2=0x7fb19015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1900b1650 0x7fb19015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 -- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb19006b0d0 msgr2=0x7fb1900c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb19006b0d0 0x7fb1900c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 -- 172.21.15.79:0/4164352370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800094a0 con 0x7fb190160080 2023-06-08T03:50:21.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb195ca7700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb19006b0d0 0x7fb1900c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1964a8700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 0x7fb19009af80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb18800cb10 tx=0x7fb18800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188003c00 con 0x7fb190160080 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb188003d60 con 0x7fb190160080 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1900ad840 con 0x7fb190160080 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18800aae0 con 0x7fb190160080 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.759+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1900ade00 con 0x7fb190160080 2023-06-08T03:50:21.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.760+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb190073e90 con 0x7fb190160080 2023-06-08T03:50:21.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.761+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb18800ac40 con 0x7fb190160080 2023-06-08T03:50:21.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.761+0000 7fb1867fc700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb17406ace0 0x7fb17406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:21.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.761+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb188089080 con 0x7fb190160080 2023-06-08T03:50:21.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.761+0000 7fb195ca7700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb17406ace0 0x7fb17406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:21.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.762+0000 7fb195ca7700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb17406ace0 0x7fb17406d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb180000c00 tx=0x7fb18001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:21.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.765+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb188054480 con 0x7fb190160080 2023-06-08T03:50:21.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.954+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb19006e200 con 0x7fb190160080 2023-06-08T03:50:21.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.954+0000 7fb1867fc700 1 -- 172.21.15.79:0/4164352370 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb188057aa0 con 0x7fb190160080 2023-06-08T03:50:21.957 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": "rbd" 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 2, 2023-06-08T03:50:21.958 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2023-06-08T03:50:21.959 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:50:21.959 INFO:teuthology.orchestra.run.smithi079.stdout:] 2023-06-08T03:50:21.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb17406ace0 msgr2=0x7fb17406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb17406ace0 0x7fb17406d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb180000c00 tx=0x7fb18001c040 comp rx=0 tx=0).stop 2023-06-08T03:50:21.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 msgr2=0x7fb19009af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:21.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 0x7fb19009af80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb18800cb10 tx=0x7fb18800ce20 comp rx=0 tx=0).stop 2023-06-08T03:50:21.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 shutdown_connections 2023-06-08T03:50:21.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb190160080 0x7fb19009af80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb19006b0d0 0x7fb1900c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb17406ace0 0x7fb17406d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 --2- 172.21.15.79:0/4164352370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb1900b1650 0x7fb19015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:21.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.956+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 >> 172.21.15.79:0/4164352370 conn(0x7fb19006a670 msgr2=0x7fb1900cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:21.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.957+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 shutdown_connections 2023-06-08T03:50:21.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:21.957+0000 7fb197f0b700 1 -- 172.21.15.79:0/4164352370 wait complete. 2023-06-08T03:50:21.971 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-08T03:50:21.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-08T03:50:22.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.092+0000 7fdb9c8a8700 1 Processor -- start 2023-06-08T03:50:22.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.092+0000 7fdb9c8a8700 1 -- start start 2023-06-08T03:50:22.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb9c8a8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdb98158d00 0x7fdb980ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb9c8a8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9815c9c0 con 0x7fdb980b16d0 2023-06-08T03:50:22.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb9c8a8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9815cca0 con 0x7fdb980cd540 2023-06-08T03:50:22.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb9c8a8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9815d640 con 0x7fdb98158d00 2023-06-08T03:50:22.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb96d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdb980b16d0 0x7fdb980ca2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57744/0 (socket says 172.21.15.79:57744) 2023-06-08T03:50:22.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb96d9d700 1 -- 172.21.15.79:0/3187457512 learned_addr learned my addr 172.21.15.79:0/3187457512 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:22.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901719377 0 0) 0x7fdb9815c9c0 con 0x7fdb980b16d0 2023-06-08T03:50:22.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.093+0000 7fdb9659c700 1 --2- 172.21.15.79:0/3187457512 >> v2:172.21.15.156:3300/0 conn(0x7fdb98158d00 0x7fdb980ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb800035b0 con 0x7fdb980b16d0 2023-06-08T03:50:22.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600795201 0 0) 0x7fdb800035b0 con 0x7fdb980b16d0 2023-06-08T03:50:22.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 >> v2:172.21.15.156:3300/0 conn(0x7fdb98158d00 msgr2=0x7fdb980ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:22.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 --2- 172.21.15.79:0/3187457512 >> v2:172.21.15.156:3300/0 conn(0x7fdb98158d00 0x7fdb980ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:22.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 >> v1:172.21.15.156:6789/0 conn(0x7fdb980cd540 legacy=0x7fdb980b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:22.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9815cf80 con 0x7fdb980b16d0 2023-06-08T03:50:22.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fdb900019f0 con 0x7fdb980b16d0 2023-06-08T03:50:22.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 >> v1:172.21.15.79:6789/0 conn(0x7fdb980b16d0 legacy=0x7fdb980ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80003a00 0x7fdb80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb80004850 0x7fdb80004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb80005320 0x7fdb8000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9815d640 con 0x7fdb80004850 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8000c250 con 0x7fdb80003a00 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8000c410 con 0x7fdb80005320 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb80004850 0x7fdb80004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9659c700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80003a00 0x7fdb80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb96d9d700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb80005320 0x7fdb8000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 -- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb80005320 msgr2=0x7fdb8000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb80005320 0x7fdb8000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 -- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80003a00 msgr2=0x7fdb80003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80003a00 0x7fdb80003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 -- 172.21.15.79:0/3187457512 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800035b0 con 0x7fdb80004850 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb96d9d700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb80005320 0x7fdb8000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:22.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb96d9d700 1 -- 172.21.15.79:0/3187457512 reap_dead start 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb80004850 0x7fdb80004cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0017c0 tx=0x7fdb80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1635edba043d3e33 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0052d0 con 0x7fdb80004850 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c005430 con 0x7fdb80004850 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.094+0000 7fdb9559a700 1 -- 172.21.15.79:0/3187457512 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0038c0 con 0x7fdb80004850 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/3187457512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb80004850 msgr2=0x7fdb80004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb80004850 0x7fdb80004cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0017c0 tx=0x7fdb80005880 comp rx=0 tx=0).stop 2023-06-08T03:50:22.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/3187457512 shutdown_connections 2023-06-08T03:50:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/3187457512 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb80004850 0x7fdb80004cc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/3187457512 >> 172.21.15.79:0/3187457512 conn(0x7fdb980cbfc0 msgr2=0x7fdb980cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/3187457512 shutdown_connections 2023-06-08T03:50:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.095+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/3187457512 wait complete. 2023-06-08T03:50:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.096+0000 7fdb9c8a8700 1 Processor -- start 2023-06-08T03:50:22.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.096+0000 7fdb9c8a8700 1 -- start start 2023-06-08T03:50:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9c8a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80004850 0x7fdb98151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9c8a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 0x7fdb981528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9c8a8700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb980b27e0 0x7fdb98152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9c8a8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb98151a40 con 0x7fdb980b2370 2023-06-08T03:50:22.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9c8a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980c6840 con 0x7fdb80004850 2023-06-08T03:50:22.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9c8a8700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb980b4c40 con 0x7fdb980b27e0 2023-06-08T03:50:22.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb95d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 0x7fdb981528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb95d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 0x7fdb981528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34278/0 (socket says 172.21.15.79:34278) 2023-06-08T03:50:22.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb95d9b700 1 -- 172.21.15.79:0/7857988 learned_addr learned my addr 172.21.15.79:0/7857988 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:22.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb9659c700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80004850 0x7fdb98151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb95d9b700 1 -- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb980b27e0 msgr2=0x7fdb98152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:22.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.097+0000 7fdb96d9d700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb980b27e0 0x7fdb98152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb980b27e0 0x7fdb98152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb95d9b700 1 -- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80004850 msgr2=0x7fdb98151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80004850 0x7fdb98151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:22.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb95d9b700 1 -- 172.21.15.79:0/7857988 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980b4f20 con 0x7fdb980b2370 2023-06-08T03:50:22.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb9659c700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80004850 0x7fdb98151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:22.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb95d9b700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 0x7fdb981528c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c015040 tx=0x7fdb8c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:22.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c001290 con 0x7fdb980b2370 2023-06-08T03:50:22.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c006860 con 0x7fdb980b2370 2023-06-08T03:50:22.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c006bf0 con 0x7fdb980b2370 2023-06-08T03:50:22.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb980c6ae0 con 0x7fdb980b2370 2023-06-08T03:50:22.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.098+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb980b5200 con 0x7fdb980b2370 2023-06-08T03:50:22.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.099+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb980c6e50 con 0x7fdb980b2370 2023-06-08T03:50:22.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.099+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb8c019040 con 0x7fdb980b2370 2023-06-08T03:50:22.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.100+0000 7fdb86ffd700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdb7806ace0 0x7fdb7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:22.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.100+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdb8c089c10 con 0x7fdb980b2370 2023-06-08T03:50:22.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.100+0000 7fdb9659c700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdb7806ace0 0x7fdb7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:22.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.100+0000 7fdb9659c700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdb7806ace0 0x7fdb7806d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdb98157f50 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:22.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.104+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb8c0550a0 con 0x7fdb980b2370 2023-06-08T03:50:22.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:22.294+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdb98073e90 con 0x7fdb980b2370 2023-06-08T03:50:23.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.263+0000 7fdb86ffd700 1 -- 172.21.15.79:0/7857988 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdb8c01b370 con 0x7fdb980b2370 2023-06-08T03:50:23.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.269+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdb7806ace0 msgr2=0x7fdb7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:23.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.269+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdb7806ace0 0x7fdb7806d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdb98157f50 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2023-06-08T03:50:23.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.269+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 msgr2=0x7fdb981528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:23.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.269+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 0x7fdb981528c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c015040 tx=0x7fdb8c001120 comp rx=0 tx=0).stop 2023-06-08T03:50:23.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 shutdown_connections 2023-06-08T03:50:23.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdb7806ace0 0x7fdb7806d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdb980b27e0 0x7fdb98152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdb80004850 0x7fdb98151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 --2- 172.21.15.79:0/7857988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb980b2370 0x7fdb981528c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 >> 172.21.15.79:0/7857988 conn(0x7fdb980cbfc0 msgr2=0x7fdb9806b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:23.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 shutdown_connections 2023-06-08T03:50:23.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.270+0000 7fdb9c8a8700 1 -- 172.21.15.79:0/7857988 wait complete. 2023-06-08T03:50:23.274 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2023-06-08T03:50:23.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-06-08T03:50:23.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.415+0000 7fad2cd7c700 1 Processor -- start 2023-06-08T03:50:23.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.415+0000 7fad2cd7c700 1 -- start start 2023-06-08T03:50:23.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2cd7c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fad280cce80 0x7fad28070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2cd7c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fad2815fb10 0x7fad280cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2cd7c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280b20f0 0x7fad280b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2cd7c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad28070bd0 con 0x7fad2815fb10 2023-06-08T03:50:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2cd7c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280cad50 con 0x7fad280cce80 2023-06-08T03:50:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2cd7c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280ce570 con 0x7fad280b20f0 2023-06-08T03:50:23.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fad2815fb10 0x7fad280cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fad2815fb10 0x7fad280cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34286/0 (socket says 172.21.15.79:34286) 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 -- 172.21.15.79:0/1858904521 learned_addr learned my addr 172.21.15.79:0/1858904521 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad2659c700 1 --2- 172.21.15.79:0/1858904521 >> v2:172.21.15.156:3300/0 conn(0x7fad280cce80 0x7fad28070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad26d9d700 1 --2- 172.21.15.79:0/1858904521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280b20f0 0x7fad280b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 -- 172.21.15.79:0/1858904521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280b20f0 msgr2=0x7fad280b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 --2- 172.21.15.79:0/1858904521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280b20f0 0x7fad280b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 -- 172.21.15.79:0/1858904521 >> v2:172.21.15.156:3300/0 conn(0x7fad280cce80 msgr2=0x7fad28070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:23.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 --2- 172.21.15.79:0/1858904521 >> v2:172.21.15.156:3300/0 conn(0x7fad280cce80 0x7fad28070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.416+0000 7fad25d9b700 1 -- 172.21.15.79:0/1858904521 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280cd790 con 0x7fad2815fb10 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad25d9b700 1 --2- 172.21.15.79:0/1858904521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad280cd250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad2806ab40 tx=0x7fad18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ade6d457668f83 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2559a700 1 -- 172.21.15.79:0/1858904521 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011830 con 0x7fad2815fb10 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2559a700 1 -- 172.21.15.79:0/1858904521 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18011990 con 0x7fad2815fb10 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2559a700 1 -- 172.21.15.79:0/1858904521 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011d20 con 0x7fad2815fb10 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 -- 172.21.15.79:0/1858904521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 msgr2=0x7fad280cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:23.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/1858904521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad280cd250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad2806ab40 tx=0x7fad18009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 -- 172.21.15.79:0/1858904521 shutdown_connections 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/1858904521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280b20f0 0x7fad280b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/1858904521 >> v2:172.21.15.156:3300/0 conn(0x7fad280cce80 0x7fad28070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/1858904521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad280cd250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 -- 172.21.15.79:0/1858904521 >> 172.21.15.79:0/1858904521 conn(0x7fad2806e1e0 msgr2=0x7fad2806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 -- 172.21.15.79:0/1858904521 shutdown_connections 2023-06-08T03:50:23.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.417+0000 7fad2cd7c700 1 -- 172.21.15.79:0/1858904521 wait complete. 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.419+0000 7fad2cd7c700 1 Processor -- start 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.419+0000 7fad2cd7c700 1 -- start start 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.419+0000 7fad2cd7c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad280b20f0 0x7fad280c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.419+0000 7fad2cd7c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280cce80 0x7fad281e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.419+0000 7fad2cd7c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad2815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.419+0000 7fad2cd7c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281e6420 con 0x7fad2815fb10 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad2cd7c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281e5af0 con 0x7fad280b20f0 2023-06-08T03:50:23.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad2cd7c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad281e5dd0 con 0x7fad280cce80 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad2815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad2815cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34294/0 (socket says 172.21.15.79:34294) 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 -- 172.21.15.79:0/2691211363 learned_addr learned my addr 172.21.15.79:0/2691211363 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad25d9b700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280cce80 0x7fad281e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad2659c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad280b20f0 0x7fad280c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 -- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280cce80 msgr2=0x7fad281e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280cce80 0x7fad281e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 -- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad280b20f0 msgr2=0x7fad280c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad280b20f0 0x7fad280c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 -- 172.21.15.79:0/2691211363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18009550 con 0x7fad2815fb10 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad26d9d700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad2815cc40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad20007d20 tx=0x7fad2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000ede0 con 0x7fad2815fb10 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20014420 con 0x7fad2815fb10 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad281e60b0 con 0x7fad2815fb10 2023-06-08T03:50:23.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200147d0 con 0x7fad2815fb10 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.420+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281f17f0 con 0x7fad2815fb10 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.421+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad281e6590 con 0x7fad2815fb10 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.422+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad20014930 con 0x7fad2815fb10 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.422+0000 7fad16ffd700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fad0806ace0 0x7fad0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.422+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fad20089910 con 0x7fad2815fb10 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.422+0000 7fad2659c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fad0806ace0 0x7fad0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.423+0000 7fad2659c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fad0806ace0 0x7fad0806d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009a10 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:23.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.427+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fad20054c30 con 0x7fad2815fb10 2023-06-08T03:50:23.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:23.612+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fad28073e90 con 0x7fad2815fb10 2023-06-08T03:50:24.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.271+0000 7fad16ffd700 1 -- 172.21.15.79:0/2691211363 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fad20058250 con 0x7fad2815fb10 2023-06-08T03:50:24.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.277+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fad0806ace0 msgr2=0x7fad0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.277+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fad0806ace0 0x7fad0806d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009a10 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2023-06-08T03:50:24.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.277+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 msgr2=0x7fad2815cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.277+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad2815cc40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fad20007d20 tx=0x7fad2000e3c0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.283+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 shutdown_connections 2023-06-08T03:50:24.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.283+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad2815fb10 0x7fad2815cc40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.283+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad280b20f0 0x7fad280c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.286 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.283+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fad0806ace0 0x7fad0806d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.283+0000 7fad2cd7c700 1 --2- 172.21.15.79:0/2691211363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad280cce80 0x7fad281e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.283+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 >> 172.21.15.79:0/2691211363 conn(0x7fad2806e1e0 msgr2=0x7fad280c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:24.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.287+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 shutdown_connections 2023-06-08T03:50:24.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.288+0000 7fad2cd7c700 1 -- 172.21.15.79:0/2691211363 wait complete. 2023-06-08T03:50:24.292 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2023-06-08T03:50:24.304 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-08T03:50:24.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.427+0000 7f76e38e5700 1 Processor -- start 2023-06-08T03:50:24.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.428+0000 7f76e38e5700 1 -- start start 2023-06-08T03:50:24.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.428+0000 7f76e38e5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f76dc15c1f0 0x7f76dc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.428+0000 7f76e38e5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.428+0000 7f76e38e5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76dc0cd140 con 0x7f76dc071fe0 2023-06-08T03:50:24.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.428+0000 7f76e38e5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc06a710 con 0x7f76dc15c1f0 2023-06-08T03:50:24.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.428+0000 7f76e38e5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc071110 con 0x7f76dc069d50 2023-06-08T03:50:24.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e0e80700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f76dc071fe0 0x7f76dc0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57774/0 (socket says 172.21.15.79:57774) 2023-06-08T03:50:24.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e0e80700 1 -- 172.21.15.79:0/2056261536 learned_addr learned my addr 172.21.15.79:0/2056261536 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:24.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1e82700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1681700 1 --2- 172.21.15.79:0/2056261536 >> v2:172.21.15.156:3300/0 conn(0x7f76dc15c1f0 0x7f76dc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76d3fff700 1 -- 172.21.15.79:0/2056261536 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188281050 0 0) 0x7f76dc0cd140 con 0x7f76dc071fe0 2023-06-08T03:50:24.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76d3fff700 1 -- 172.21.15.79:0/2056261536 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c80035b0 con 0x7f76dc071fe0 2023-06-08T03:50:24.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76d3fff700 1 -- 172.21.15.79:0/2056261536 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426955187 0 0) 0x7f76c80035b0 con 0x7f76dc071fe0 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1681700 1 -- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 msgr2=0x7f76dc06a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1681700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc06a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1681700 1 -- 172.21.15.79:0/2056261536 >> v1:172.21.15.79:6789/0 conn(0x7f76dc071fe0 legacy=0x7f76dc0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1681700 1 -- 172.21.15.79:0/2056261536 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc071be0 con 0x7f76dc15c1f0 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1e82700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.429+0000 7f76e1681700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc15c1f0 0x7f76dc06d8a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009840 tx=0x7f76cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac87f0eaef96a318 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:24.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76d3fff700 1 -- 172.21.15.79:0/2056261536 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8001670 con 0x7f76dc15c1f0 2023-06-08T03:50:24.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76d3fff700 1 -- 172.21.15.79:0/2056261536 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc004030 con 0x7f76dc15c1f0 2023-06-08T03:50:24.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76d3fff700 1 -- 172.21.15.79:0/2056261536 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0043c0 con 0x7f76dc15c1f0 2023-06-08T03:50:24.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 -- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc15c1f0 msgr2=0x7f76dc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc15c1f0 0x7f76dc06d8a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009840 tx=0x7f76cc009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:24.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 -- 172.21.15.79:0/2056261536 shutdown_connections 2023-06-08T03:50:24.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 --2- 172.21.15.79:0/2056261536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc15c1f0 0x7f76dc06d8a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 -- 172.21.15.79:0/2056261536 >> 172.21.15.79:0/2056261536 conn(0x7f76dc0b1d30 msgr2=0x7f76dc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:24.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 -- 172.21.15.79:0/2056261536 shutdown_connections 2023-06-08T03:50:24.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.430+0000 7f76e38e5700 1 -- 172.21.15.79:0/2056261536 wait complete. 2023-06-08T03:50:24.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.431+0000 7f76e38e5700 1 Processor -- start 2023-06-08T03:50:24.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.431+0000 7f76e38e5700 1 -- start start 2023-06-08T03:50:24.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e38e5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e38e5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc071fe0 0x7f76dc156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e38e5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 0x7f76dc158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e38e5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc157c80 con 0x7f76dc15c1f0 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e38e5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc153010 con 0x7f76dc071fe0 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e38e5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc156dd0 con 0x7f76dc069d50 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 0x7f76dc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 0x7f76dc158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34308/0 (socket says 172.21.15.79:34308) 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 -- 172.21.15.79:0/3444459351 learned_addr learned my addr 172.21.15.79:0/3444459351 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:24.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1681700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e0e80700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc071fe0 0x7f76dc156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 -- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 msgr2=0x7f76dc152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 -- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc071fe0 msgr2=0x7f76dc156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc071fe0 0x7f76dc156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.432+0000 7f76e1e82700 1 -- 172.21.15.79:0/3444459351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4009a70 con 0x7f76dc15c1f0 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.433+0000 7f76e1e82700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 0x7f76dc158110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009e60 tx=0x7f76d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:24.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.433+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4009cd0 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.433+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d400f040 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.433+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc0094a0 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.433+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d400ccb0 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.433+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc157350 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.434+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc157e30 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.434+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f76d4014480 con 0x7f76dc15c1f0 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.435+0000 7f76d1ffb700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f76c006ace0 0x7f76c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.435+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f76d4089a00 con 0x7f76dc15c1f0 2023-06-08T03:50:24.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.435+0000 7f76e1681700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f76c006ace0 0x7f76c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.435+0000 7f76e1681700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f76c006ace0 0x7f76c006d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0090b0 tx=0x7f76cc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:24.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.439+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f76dc157e30 con 0x7f76dc15c1f0 2023-06-08T03:50:24.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.667+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f76dc157e30 con 0x7f76dc15c1f0 2023-06-08T03:50:24.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.674+0000 7f76d1ffb700 1 -- 172.21.15.79:0/3444459351 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f76dc157e30 con 0x7f76dc15c1f0 2023-06-08T03:50:24.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f76c006ace0 msgr2=0x7f76c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f76c006ace0 0x7f76c006d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0090b0 tx=0x7f76cc008030 comp rx=0 tx=0).stop 2023-06-08T03:50:24.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 msgr2=0x7f76dc158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 0x7f76dc158110 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009e60 tx=0x7f76d400c3a0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 shutdown_connections 2023-06-08T03:50:24.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f76dc15c1f0 0x7f76dc158110 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f76c006ace0 0x7f76c006d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f76dc069d50 0x7f76dc152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 --2- 172.21.15.79:0/3444459351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76dc071fe0 0x7f76dc156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 >> 172.21.15.79:0/3444459351 conn(0x7f76dc0b1d30 msgr2=0x7f76dc0cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:24.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.679+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 shutdown_connections 2023-06-08T03:50:24.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.680+0000 7f76e38e5700 1 -- 172.21.15.79:0/3444459351 wait complete. 2023-06-08T03:50:24.686 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 3 and data pool 4 2023-06-08T03:50:24.707 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-06-08T03:50:24.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-06-08T03:50:24.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.827+0000 7f2747078700 1 Processor -- start 2023-06-08T03:50:24.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.827+0000 7f2747078700 1 -- start start 2023-06-08T03:50:24.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2747078700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27400b1ea0 0x7f27400d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2747078700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f27400ce2d0 0x7f27400d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2747078700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27400d3880 con 0x7f274006cbd0 2023-06-08T03:50:24.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2747078700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27400d2d00 con 0x7f27400ce2d0 2023-06-08T03:50:24.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2747078700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27400d2fe0 con 0x7f27400b1ea0 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f273ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f274006cbd0 0x7f274006b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57800/0 (socket says 172.21.15.79:57800) 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f273ffff700 1 -- 172.21.15.79:0/2277978976 learned_addr learned my addr 172.21.15.79:0/2277978976 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2744e14700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27400b1ea0 0x7f27400d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.828+0000 7f2745615700 1 --2- 172.21.15.79:0/2277978976 >> v2:172.21.15.156:3300/0 conn(0x7f27400ce2d0 0x7f27400d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f273f7fe700 1 -- 172.21.15.79:0/2277978976 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828138022 0 0) 0x7f27400d3880 con 0x7f274006cbd0 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f273f7fe700 1 -- 172.21.15.79:0/2277978976 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27300035b0 con 0x7f274006cbd0 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f2745615700 1 -- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27400b1ea0 msgr2=0x7f27400d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:24.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f2745615700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27400b1ea0 0x7f27400d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f2745615700 1 -- 172.21.15.79:0/2277978976 >> v1:172.21.15.79:6789/0 conn(0x7f274006cbd0 legacy=0x7f274006b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f2745615700 1 -- 172.21.15.79:0/2277978976 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27400d32c0 con 0x7f27400ce2d0 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f2744e14700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27400b1ea0 0x7f27400d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f2745615700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400ce2d0 0x7f27400d0a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2738009dc0 tx=0x7f273800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c645e98716cb68a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f273f7fe700 1 -- 172.21.15.79:0/2277978976 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800e070 con 0x7f27400ce2d0 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f273f7fe700 1 -- 172.21.15.79:0/2277978976 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273800bae0 con 0x7f27400ce2d0 2023-06-08T03:50:24.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.829+0000 7f273f7fe700 1 -- 172.21.15.79:0/2277978976 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800be90 con 0x7f27400ce2d0 2023-06-08T03:50:24.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 -- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400ce2d0 msgr2=0x7f27400d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400ce2d0 0x7f27400d0a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2738009dc0 tx=0x7f273800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 -- 172.21.15.79:0/2277978976 shutdown_connections 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400ce2d0 0x7f27400d0a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 --2- 172.21.15.79:0/2277978976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27400b1ea0 0x7f27400d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 -- 172.21.15.79:0/2277978976 >> 172.21.15.79:0/2277978976 conn(0x7f2740071da0 msgr2=0x7f274009bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 -- 172.21.15.79:0/2277978976 shutdown_connections 2023-06-08T03:50:24.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.830+0000 7f2747078700 1 -- 172.21.15.79:0/2277978976 wait complete. 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.831+0000 7f2747078700 1 Processor -- start 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.831+0000 7f2747078700 1 -- start start 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2747078700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f274006cbd0 0x7f274015eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2747078700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400b1ea0 0x7f27400c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2747078700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 0x7f27400c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2747078700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400b5ec0 con 0x7f27400ce2d0 2023-06-08T03:50:24.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2747078700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274015f0c0 con 0x7f27400b1ea0 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2747078700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27400cc260 con 0x7f274006cbd0 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 0x7f27400c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 0x7f27400c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34332/0 (socket says 172.21.15.79:34332) 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 -- 172.21.15.79:0/1294462816 learned_addr learned my addr 172.21.15.79:0/1294462816 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f273ffff700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400b1ea0 0x7f27400c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2744e14700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f274006cbd0 0x7f274015eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 -- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f274006cbd0 msgr2=0x7f274015eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f274006cbd0 0x7f274015eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 -- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400b1ea0 msgr2=0x7f27400c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400b1ea0 0x7f27400c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 -- 172.21.15.79:0/1294462816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2734009430 con 0x7f27400ce2d0 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f2745615700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 0x7f27400c6990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2740066900 tx=0x7f2738004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.832+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380044d0 con 0x7f27400ce2d0 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.833+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738020070 con 0x7f27400ce2d0 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.833+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2738009a70 con 0x7f27400ce2d0 2023-06-08T03:50:24.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.833+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738004870 con 0x7f27400ce2d0 2023-06-08T03:50:24.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.833+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27400b63f0 con 0x7f27400ce2d0 2023-06-08T03:50:24.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.834+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2740073e90 con 0x7f27400ce2d0 2023-06-08T03:50:24.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.834+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2738012430 con 0x7f27400ce2d0 2023-06-08T03:50:24.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.834+0000 7f273d7fa700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f272406ace0 0x7f272406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:24.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.834+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2738022070 con 0x7f27400ce2d0 2023-06-08T03:50:24.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.834+0000 7f2744e14700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f272406ace0 0x7f272406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:24.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.835+0000 7f2744e14700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f272406ace0 0x7f272406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2734009d50 tx=0x7f27340091c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:24.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:24.838+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f27380560e0 con 0x7f27400ce2d0 2023-06-08T03:50:25.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.036+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f27400cfa80 con 0x7f27400ce2d0 2023-06-08T03:50:25.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.681+0000 7f273d7fa700 1 -- 172.21.15.79:0/1294462816 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2738059700 con 0x7f27400ce2d0 2023-06-08T03:50:25.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f272406ace0 msgr2=0x7f272406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:25.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f272406ace0 0x7f272406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2734009d50 tx=0x7f27340091c0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 msgr2=0x7f27400c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 0x7f27400c6990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2740066900 tx=0x7f2738004ad0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 shutdown_connections 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27400ce2d0 0x7f27400c6990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f272406ace0 0x7f272406d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f274006cbd0 0x7f274015eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 --2- 172.21.15.79:0/1294462816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27400b1ea0 0x7f27400c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 >> 172.21.15.79:0/1294462816 conn(0x7f2740071da0 msgr2=0x7f27400b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:25.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 shutdown_connections 2023-06-08T03:50:25.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.683+0000 7f2747078700 1 -- 172.21.15.79:0/1294462816 wait complete. 2023-06-08T03:50:25.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-06-08T03:50:25.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.825+0000 7fcb27f6b700 1 Processor -- start 2023-06-08T03:50:25.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.825+0000 7fcb27f6b700 1 -- start start 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb27f6b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb2006d8a0 0x7fcb20071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb27f6b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcb20069d50 0x7fcb2006a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb27f6b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb200cd140 con 0x7fcb2015c1f0 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb27f6b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2006a730 con 0x7fcb2006d8a0 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb27f6b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb20071110 con 0x7fcb20069d50 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25506700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb2006d8a0 0x7fcb20071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:25.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25d07700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fcb2015c1f0 0x7fcb200cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57812/0 (socket says 172.21.15.79:57812) 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25d07700 1 -- 172.21.15.79:0/3235717146 learned_addr learned my addr 172.21.15.79:0/3235717146 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb26508700 1 --2- 172.21.15.79:0/3235717146 >> v2:172.21.15.156:3300/0 conn(0x7fcb20069d50 0x7fcb2006a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25506700 1 -- 172.21.15.79:0/3235717146 >> v2:172.21.15.156:3300/0 conn(0x7fcb20069d50 msgr2=0x7fcb2006a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25506700 1 --2- 172.21.15.79:0/3235717146 >> v2:172.21.15.156:3300/0 conn(0x7fcb20069d50 0x7fcb2006a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25506700 1 -- 172.21.15.79:0/3235717146 >> v1:172.21.15.79:6789/0 conn(0x7fcb2015c1f0 legacy=0x7fcb200cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb25506700 1 -- 172.21.15.79:0/3235717146 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb20071be0 con 0x7fcb2006d8a0 2023-06-08T03:50:25.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb24d05700 1 -- 172.21.15.79:0/3235717146 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527753069 0 0) 0x7fcb200cd140 con 0x7fcb2015c1f0 2023-06-08T03:50:25.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.826+0000 7fcb26508700 1 --2- 172.21.15.79:0/3235717146 >> v2:172.21.15.156:3300/0 conn(0x7fcb20069d50 0x7fcb2006a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:25.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb25506700 1 --2- 172.21.15.79:0/3235717146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb20071fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb20070e40 tx=0x7fcb1c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e243c2521681fc server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:25.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb24d05700 1 -- 172.21.15.79:0/3235717146 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c011830 con 0x7fcb2006d8a0 2023-06-08T03:50:25.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb24d05700 1 -- 172.21.15.79:0/3235717146 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c011990 con 0x7fcb2006d8a0 2023-06-08T03:50:25.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb24d05700 1 -- 172.21.15.79:0/3235717146 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c011d20 con 0x7fcb2006d8a0 2023-06-08T03:50:25.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 -- 172.21.15.79:0/3235717146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 msgr2=0x7fcb20071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:25.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/3235717146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb20071fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb20070e40 tx=0x7fcb1c009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:25.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 -- 172.21.15.79:0/3235717146 shutdown_connections 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/3235717146 >> v2:172.21.15.156:3300/0 conn(0x7fcb20069d50 0x7fcb2006a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/3235717146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb20071fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 -- 172.21.15.79:0/3235717146 >> 172.21.15.79:0/3235717146 conn(0x7fcb200b1d30 msgr2=0x7fcb2006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 -- 172.21.15.79:0/3235717146 shutdown_connections 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.827+0000 7fcb27f6b700 1 -- 172.21.15.79:0/3235717146 wait complete. 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.829+0000 7fcb27f6b700 1 Processor -- start 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.829+0000 7fcb27f6b700 1 -- start start 2023-06-08T03:50:25.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb20069d50 0x7fcb20153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb200c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2015c1f0 0x7fcb200b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20153b40 con 0x7fcb2006d8a0 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200c82c0 con 0x7fcb2015c1f0 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200c8e40 con 0x7fcb20069d50 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb200c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:25.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb200c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34344/0 (socket says 172.21.15.79:34344) 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 -- 172.21.15.79:0/2890697921 learned_addr learned my addr 172.21.15.79:0/2890697921 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb26508700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2015c1f0 0x7fcb200b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 -- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb20069d50 msgr2=0x7fcb20153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb20069d50 0x7fcb20153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 -- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2015c1f0 msgr2=0x7fcb200b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2015c1f0 0x7fcb200b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 -- 172.21.15.79:0/2890697921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c0094a0 con 0x7fcb2006d8a0 2023-06-08T03:50:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb25506700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb200c92d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c014040 tx=0x7fcb1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:25.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0038a0 con 0x7fcb2006d8a0 2023-06-08T03:50:25.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c01e070 con 0x7fcb2006d8a0 2023-06-08T03:50:25.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb200c8510 con 0x7fcb2006d8a0 2023-06-08T03:50:25.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c003c60 con 0x7fcb2006d8a0 2023-06-08T03:50:25.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.830+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb200c8ad0 con 0x7fcb2006d8a0 2023-06-08T03:50:25.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.831+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb200c9120 con 0x7fcb2006d8a0 2023-06-08T03:50:25.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.832+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb1c02d450 con 0x7fcb2006d8a0 2023-06-08T03:50:25.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.832+0000 7fcb167fc700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcb0406ace0 0x7fcb0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:25.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.832+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6288+0+0 (secure 0 0 0) 0x7fcb1c08a730 con 0x7fcb2006d8a0 2023-06-08T03:50:25.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.832+0000 7fcb25d07700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcb0406ace0 0x7fcb0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.833+0000 7fcb25d07700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcb0406ace0 0x7fcb0406d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb10001610 tx=0x7fcb100079f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:25.836+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb1c055790 con 0x7fcb2006d8a0 2023-06-08T03:50:26.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:26.027+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcb200c8f80 con 0x7fcb2006d8a0 2023-06-08T03:50:27.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.896+0000 7fcb167fc700 1 -- 172.21.15.79:0/2890697921 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fcb200c8f80 con 0x7fcb2006d8a0 2023-06-08T03:50:27.908 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcb0406ace0 msgr2=0x7fcb0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:27.908 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcb0406ace0 0x7fcb0406d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb10001610 tx=0x7fcb100079f0 comp rx=0 tx=0).stop 2023-06-08T03:50:27.908 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 msgr2=0x7fcb200c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb200c92d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c014040 tx=0x7fcb1c004750 comp rx=0 tx=0).stop 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 shutdown_connections 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2015c1f0 0x7fcb200b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcb0406ace0 0x7fcb0406d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb20069d50 0x7fcb20153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 --2- 172.21.15.79:0/2890697921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb2006d8a0 0x7fcb200c92d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:27.909 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 >> 172.21.15.79:0/2890697921 conn(0x7fcb200b1d30 msgr2=0x7fcb200b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 shutdown_connections 2023-06-08T03:50:27.910 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:27.904+0000 7fcb27f6b700 1 -- 172.21.15.79:0/2890697921 wait complete. 2023-06-08T03:50:27.915 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data_ec' created 2023-06-08T03:50:27.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-06-08T03:50:28.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 Processor -- start 2023-06-08T03:50:28.058 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 -- start start 2023-06-08T03:50:28.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65280ccea0 0x7f652806a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 0x7f65280707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652806a260 con 0x7f652806ac30 2023-06-08T03:50:28.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6528142370 con 0x7f65280ccea0 2023-06-08T03:50:28.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.054+0000 7f652eb24700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652806d340 con 0x7f65280cd270 2023-06-08T03:50:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f6527fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f652806ac30 0x7f652806c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57830/0 (socket says 172.21.15.79:57830) 2023-06-08T03:50:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f6527fff700 1 -- 172.21.15.79:0/1495001091 learned_addr learned my addr 172.21.15.79:0/1495001091 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:28.062 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 0x7f65280707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652c8c0700 1 --2- 172.21.15.79:0/1495001091 >> v2:172.21.15.156:3300/0 conn(0x7f65280ccea0 0x7f652806a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883107430 0 0) 0x7f652806a260 con 0x7f652806ac30 2023-06-08T03:50:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65180035b0 con 0x7f652806ac30 2023-06-08T03:50:28.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925275716 0 0) 0x7f65180035b0 con 0x7f652806ac30 2023-06-08T03:50:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 msgr2=0x7f65280707f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 0x7f65280707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 >> v2:172.21.15.156:3300/0 conn(0x7f65280ccea0 msgr2=0x7f652806a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 --2- 172.21.15.79:0/1495001091 >> v2:172.21.15.156:3300/0 conn(0x7f65280ccea0 0x7f652806a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652806d620 con 0x7f652806ac30 2023-06-08T03:50:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f6514001670 con 0x7f652806ac30 2023-06-08T03:50:28.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 >> v1:172.21.15.79:6789/0 conn(0x7f652806ac30 legacy=0x7f652806c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:28.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 0x7f6518003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6518003f80 0x7f6518004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f6518005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6528142370 con 0x7f6518004d90 2023-06-08T03:50:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652806d340 con 0x7f6518003f80 2023-06-08T03:50:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651800ba40 con 0x7f65280cd270 2023-06-08T03:50:28.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f6518005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652c8c0700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 0x7f6518003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f6527fff700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6518003f80 0x7f6518004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 -- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 msgr2=0x7f6518003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:28.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65280cd270 0x7f6518003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 -- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6518003f80 msgr2=0x7f6518004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6518003f80 0x7f6518004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.055+0000 7f652d0c1700 1 -- 172.21.15.79:0/1495001091 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65180035b0 con 0x7f6518004d90 2023-06-08T03:50:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652d0c1700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f6518005200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6520009f90 tx=0x7f6518005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85fe9b420545a1d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:28.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000eb70 con 0x7f6518004d90 2023-06-08T03:50:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000f040 con 0x7f6518004d90 2023-06-08T03:50:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f65277fe700 1 -- 172.21.15.79:0/1495001091 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520014480 con 0x7f6518004d90 2023-06-08T03:50:28.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652eb24700 1 -- 172.21.15.79:0/1495001091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 msgr2=0x7f6518005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652eb24700 1 --2- 172.21.15.79:0/1495001091 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f6518005200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6520009f90 tx=0x7f6518005880 comp rx=0 tx=0).stop 2023-06-08T03:50:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652d0c1700 1 -- 172.21.15.79:0/1495001091 reap_dead start 2023-06-08T03:50:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652eb24700 1 -- 172.21.15.79:0/1495001091 shutdown_connections 2023-06-08T03:50:28.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652eb24700 1 -- 172.21.15.79:0/1495001091 >> 172.21.15.79:0/1495001091 conn(0x7f65280ceca0 msgr2=0x7f65280cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:28.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652eb24700 1 -- 172.21.15.79:0/1495001091 shutdown_connections 2023-06-08T03:50:28.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.056+0000 7f652eb24700 1 -- 172.21.15.79:0/1495001091 wait complete. 2023-06-08T03:50:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.057+0000 7f652eb24700 1 Processor -- start 2023-06-08T03:50:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 -- start start 2023-06-08T03:50:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f65280c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6528159530 0x7f6528152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65280c3cb0 0x7f65281e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c90f0 con 0x7f6518004d90 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65280c4d60 con 0x7f65280c3cb0 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652eb24700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65281599a0 con 0x7f6528159530 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.058+0000 7f652c8c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f65280c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f65280c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34364/0 (socket says 172.21.15.79:34364) 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 -- 172.21.15.79:0/3678427656 learned_addr learned my addr 172.21.15.79:0/3678427656 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 -- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6528159530 msgr2=0x7f6528152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f6527fff700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6528159530 0x7f6528152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652d0c1700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65280c3cb0 0x7f65281e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6528159530 0x7f6528152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 -- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65280c3cb0 msgr2=0x7f65281e2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65280c3cb0 0x7f65281e2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 -- 172.21.15.79:0/3678427656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652815a450 con 0x7f6518004d90 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652c8c0700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f65280c8a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f651c011aa0 tx=0x7f651c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00f9e0 con 0x7f6518004d90 2023-06-08T03:50:28.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c00fb40 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c014440 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652815a730 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.059+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652815acf0 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.060+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6528073e90 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.061+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f651c0145a0 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.061+0000 7f65257fa700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f650c06acd0 0x7f650c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.061+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f651c089ee0 con 0x7f6518004d90 2023-06-08T03:50:28.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.061+0000 7f6527fff700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f650c06acd0 0x7f650c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:28.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.062+0000 7f6527fff700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f650c06acd0 0x7f650c06d180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f652815a240 tx=0x7f6514009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:28.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.066+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f651c054d50 con 0x7f6518004d90 2023-06-08T03:50:28.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.254+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f652806b350 con 0x7f6518004d90 2023-06-08T03:50:28.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.899+0000 7f65257fa700 1 -- 172.21.15.79:0/3678427656 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f651c058370 con 0x7f6518004d90 2023-06-08T03:50:28.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.908+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f650c06acd0 msgr2=0x7f650c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:28.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.908+0000 7f652eb24700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f650c06acd0 0x7f650c06d180 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f652815a240 tx=0x7f6514009040 comp rx=0 tx=0).stop 2023-06-08T03:50:28.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.908+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 msgr2=0x7f65280c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:28.912 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.908+0000 7f652eb24700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f65280c8a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f651c011aa0 tx=0x7f651c011e60 comp rx=0 tx=0).stop 2023-06-08T03:50:28.914 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.911+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 shutdown_connections 2023-06-08T03:50:28.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.911+0000 7f652eb24700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65280c3cb0 0x7f65281e2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.911+0000 7f652eb24700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6518004d90 0x7f65280c8a80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.911+0000 7f652eb24700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f650c06acd0 0x7f650c06d180 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.911+0000 7f652eb24700 1 --2- 172.21.15.79:0/3678427656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6528159530 0x7f6528152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:28.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.911+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 >> 172.21.15.79:0/3678427656 conn(0x7f65280ceca0 msgr2=0x7f65280b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:28.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.912+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 shutdown_connections 2023-06-08T03:50:28.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:28.912+0000 7f652eb24700 1 -- 172.21.15.79:0/3678427656 wait complete. 2023-06-08T03:50:28.919 INFO:teuthology.orchestra.run.smithi079.stderr:set pool 5 allow_ec_overwrites to true 2023-06-08T03:50:28.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-06-08T03:50:29.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.060+0000 7f6bfe599700 1 Processor -- start 2023-06-08T03:50:29.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.060+0000 7f6bfe599700 1 -- start start 2023-06-08T03:50:29.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfe599700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf815a0e0 0x7f6bf8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfe599700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6bf806a290 0x7f6bf80707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfe599700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf8071290 con 0x7f6bf8070d10 2023-06-08T03:50:29.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfe599700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf8071570 con 0x7f6bf806a290 2023-06-08T03:50:29.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfe599700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80ca4c0 con 0x7f6bf815a0e0 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf77fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6bf806a290 0x7f6bf80707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfcb36700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6bf8070d10 0x7f6bf80cacc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57846/0 (socket says 172.21.15.79:57846) 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bfcb36700 1 -- 172.21.15.79:0/2943282780 learned_addr learned my addr 172.21.15.79:0/2943282780 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf7fff700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf815a0e0 0x7f6bf8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf77fe700 1 -- 172.21.15.79:0/2943282780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf815a0e0 msgr2=0x7f6bf8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf77fe700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf815a0e0 0x7f6bf8069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf77fe700 1 -- 172.21.15.79:0/2943282780 >> v1:172.21.15.79:6789/0 conn(0x7f6bf8070d10 legacy=0x7f6bf80cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf6ffd700 1 -- 172.21.15.79:0/2943282780 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114831026 0 0) 0x7f6bf8071290 con 0x7f6bf8070d10 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf77fe700 1 -- 172.21.15.79:0/2943282780 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf80ca7a0 con 0x7f6bf806a290 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf7fff700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf815a0e0 0x7f6bf8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf77fe700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf806a290 0x7f6bf80707d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009840 tx=0x7f6be8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2fb18459a37daf server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf6ffd700 1 -- 172.21.15.79:0/2943282780 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8011830 con 0x7f6bf806a290 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf6ffd700 1 -- 172.21.15.79:0/2943282780 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be8011990 con 0x7f6bf806a290 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.061+0000 7f6bf6ffd700 1 -- 172.21.15.79:0/2943282780 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8011d20 con 0x7f6bf806a290 2023-06-08T03:50:29.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 -- 172.21.15.79:0/2943282780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf806a290 msgr2=0x7f6bf80707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf806a290 0x7f6bf80707d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009840 tx=0x7f6be8009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 -- 172.21.15.79:0/2943282780 shutdown_connections 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf815a0e0 0x7f6bf8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 --2- 172.21.15.79:0/2943282780 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf806a290 0x7f6bf80707d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 -- 172.21.15.79:0/2943282780 >> 172.21.15.79:0/2943282780 conn(0x7f6bf80ced30 msgr2=0x7f6bf80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 -- 172.21.15.79:0/2943282780 shutdown_connections 2023-06-08T03:50:29.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.062+0000 7f6bfe599700 1 -- 172.21.15.79:0/2943282780 wait complete. 2023-06-08T03:50:29.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.063+0000 7f6bfe599700 1 Processor -- start 2023-06-08T03:50:29.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.063+0000 7f6bfe599700 1 -- start start 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfe599700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf806a290 0x7f6bf80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfe599700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 0x7f6bf80b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfe599700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bf815a0e0 0x7f6bf81f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfe599700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80cd310 con 0x7f6bf8070d10 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfe599700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80cd5f0 con 0x7f6bf815a0e0 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfe599700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80cd8d0 con 0x7f6bf806a290 2023-06-08T03:50:29.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 0x7f6bf80b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 0x7f6bf80b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34376/0 (socket says 172.21.15.79:34376) 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 -- 172.21.15.79:0/436600500 learned_addr learned my addr 172.21.15.79:0/436600500 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf7fff700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf806a290 0x7f6bf80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfcb36700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bf815a0e0 0x7f6bf81f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 -- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf806a290 msgr2=0x7f6bf80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf806a290 0x7f6bf80ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 -- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bf815a0e0 msgr2=0x7f6bf81f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bf815a0e0 0x7f6bf81f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bf77fe700 1 -- 172.21.15.79:0/436600500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be80094a0 con 0x7f6bf8070d10 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.064+0000 7f6bfcb36700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bf815a0e0 0x7f6bf81f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.065+0000 7f6bf77fe700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 0x7f6bf80b4240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6be8000c00 tx=0x7f6be8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.065+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be80036a0 con 0x7f6bf8070d10 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.065+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be801e070 con 0x7f6bf8070d10 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.065+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be8003a60 con 0x7f6bf8070d10 2023-06-08T03:50:29.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.065+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf80cdbb0 con 0x7f6bf8070d10 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.065+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf80b4780 con 0x7f6bf8070d10 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.066+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf806f1d0 con 0x7f6bf8070d10 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.066+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6be8003bc0 con 0x7f6bf8070d10 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.067+0000 7f6bf4ff9700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6bd806ace0 0x7f6bd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.067+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f6be808a7b0 con 0x7f6bf8070d10 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.067+0000 7f6bf7fff700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6bd806ace0 0x7f6bd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.067+0000 7f6bf7fff700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6bd806ace0 0x7f6bd806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.071+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6be8055620 con 0x7f6bf8070d10 2023-06-08T03:50:29.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.302+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f6bf8073e90 con 0x7f6bf8070d10 2023-06-08T03:50:29.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.306+0000 7f6bf4ff9700 1 -- 172.21.15.79:0/436600500 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f6be8058c40 con 0x7f6bf8070d10 2023-06-08T03:50:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6bd806ace0 msgr2=0x7f6bd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6bd806ace0 0x7f6bd806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0008040 comp rx=0 tx=0).stop 2023-06-08T03:50:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 msgr2=0x7f6bf80b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 0x7f6bf80b4240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6be8000c00 tx=0x7f6be8004750 comp rx=0 tx=0).stop 2023-06-08T03:50:29.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 shutdown_connections 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bf815a0e0 0x7f6bf81f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6bd806ace0 0x7f6bd806d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bf806a290 0x7f6bf80ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 --2- 172.21.15.79:0/436600500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6bf8070d10 0x7f6bf80b4240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 >> 172.21.15.79:0/436600500 conn(0x7f6bf80ced30 msgr2=0x7f6bf80aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 shutdown_connections 2023-06-08T03:50:29.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.308+0000 7f6bfe599700 1 -- 172.21.15.79:0/436600500 wait complete. 2023-06-08T03:50:29.314 INFO:teuthology.orchestra.run.smithi079.stderr:added data pool 5 to fsmap 2023-06-08T03:50:29.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.439+0000 7f23760d5700 1 Processor -- start 2023-06-08T03:50:29.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.439+0000 7f23760d5700 1 -- start start 2023-06-08T03:50:29.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f23760d5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f237006a460 0x7f2370071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f23760d5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 0x7f2370071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f23760d5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23700cee20 con 0x7f237015f6c0 2023-06-08T03:50:29.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f23760d5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23700cf330 con 0x7f237006a460 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f23760d5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237006fa00 con 0x7f23700cd6c0 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236f7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f237015f6c0 0x7f2370069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57858/0 (socket says 172.21.15.79:57858) 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236f7fe700 1 -- 172.21.15.79:0/1894494770 learned_addr learned my addr 172.21.15.79:0/1894494770 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142901107 0 0) 0x7f23700cee20 con 0x7f237015f6c0 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23580035b0 con 0x7f237015f6c0 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236ffff700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 0x7f2370071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236effd700 1 --2- 172.21.15.79:0/1894494770 >> v2:172.21.15.156:3300/0 conn(0x7f237006a460 0x7f2370071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674365822 0 0) 0x7f23580035b0 con 0x7f237015f6c0 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 msgr2=0x7f2370071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 0x7f2370071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 >> v2:172.21.15.156:3300/0 conn(0x7f237006a460 msgr2=0x7f2370071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 --2- 172.21.15.79:0/1894494770 >> v2:172.21.15.156:3300/0 conn(0x7f237006a460 0x7f2370071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700cd1e0 con 0x7f237015f6c0 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f2364001670 con 0x7f237015f6c0 2023-06-08T03:50:29.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 >> v1:172.21.15.79:6789/0 conn(0x7f237015f6c0 legacy=0x7f2370069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2358003a70 0x7f2358003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2358004850 0x7f2358004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 0x7f2358005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23700cf330 con 0x7f2358004850 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237006fa00 con 0x7f2358003a70 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.440+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235800ba60 con 0x7f23700cd6c0 2023-06-08T03:50:29.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2358004850 0x7f2358004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236f7fe700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2358003a70 0x7f2358003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236ffff700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 0x7f2358005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 -- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 msgr2=0x7f2358005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23700cd6c0 0x7f2358005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 -- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2358003a70 msgr2=0x7f2358003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2358003a70 0x7f2358003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 -- 172.21.15.79:0/1894494770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580035b0 con 0x7f2358004850 2023-06-08T03:50:29.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236f7fe700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2358003a70 0x7f2358003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236effd700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2358004850 0x7f2358004cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23600098c0 tx=0x7f2358005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e81c49a992041e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000c070 con 0x7f2358004850 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360013610 con 0x7f2358004850 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236e7fc700 1 -- 172.21.15.79:0/1894494770 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600139a0 con 0x7f2358004850 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f23760d5700 1 -- 172.21.15.79:0/1894494770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2358004850 msgr2=0x7f2358004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f23760d5700 1 --2- 172.21.15.79:0/1894494770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2358004850 0x7f2358004cc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23600098c0 tx=0x7f2358005880 comp rx=0 tx=0).stop 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.441+0000 7f236ffff700 1 -- 172.21.15.79:0/1894494770 reap_dead start 2023-06-08T03:50:29.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.442+0000 7f23760d5700 1 -- 172.21.15.79:0/1894494770 shutdown_connections 2023-06-08T03:50:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.442+0000 7f23760d5700 1 -- 172.21.15.79:0/1894494770 >> 172.21.15.79:0/1894494770 conn(0x7f23700b1d20 msgr2=0x7f237006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.442+0000 7f23760d5700 1 -- 172.21.15.79:0/1894494770 shutdown_connections 2023-06-08T03:50:29.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.442+0000 7f23760d5700 1 -- 172.21.15.79:0/1894494770 wait complete. 2023-06-08T03:50:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.443+0000 7f23760d5700 1 Processor -- start 2023-06-08T03:50:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.443+0000 7f23760d5700 1 -- start start 2023-06-08T03:50:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2358004850 0x7f23701e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 0x7f2370157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237015c080 0x7f237015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237015ca30 con 0x7f2370156610 2023-06-08T03:50:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23701593f0 con 0x7f237015c080 2023-06-08T03:50:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2370159880 con 0x7f2358004850 2023-06-08T03:50:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 0x7f2370157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 0x7f2370157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34396/0 (socket says 172.21.15.79:34396) 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 -- 172.21.15.79:0/1239163670 learned_addr learned my addr 172.21.15.79:0/1239163670 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236f7fe700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2358004850 0x7f23701e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236ffff700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237015c080 0x7f237015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 -- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2358004850 msgr2=0x7f23701e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2358004850 0x7f23701e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 -- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237015c080 msgr2=0x7f237015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237015c080 0x7f237015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 -- 172.21.15.79:0/1239163670 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2370158510 con 0x7f2370156610 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236ffff700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237015c080 0x7f237015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f236effd700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 0x7f2370157fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23580057f0 tx=0x7f23600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360013b90 con 0x7f2370156610 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360013d10 con 0x7f2370156610 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23701587e0 con 0x7f2370156610 2023-06-08T03:50:29.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600038e0 con 0x7f2370156610 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.444+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2370158da0 con 0x7f2370156610 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.445+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2370073e90 con 0x7f2370156610 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.446+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2360003a40 con 0x7f2370156610 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.446+0000 7f2357fff700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f235c06ace0 0x7f235c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.446+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f236008a850 con 0x7f2370156610 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.446+0000 7f236f7fe700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f235c06ace0 0x7f235c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.447+0000 7f236f7fe700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f235c06ace0 0x7f235c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f237015c020 tx=0x7f2364007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.450+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23600556a0 con 0x7f2370156610 2023-06-08T03:50:29.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.683+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f237006dc20 con 0x7f2370156610 2023-06-08T03:50:29.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.684+0000 7f2357fff700 1 -- 172.21.15.79:0/1239163670 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f2360058cc0 con 0x7f2370156610 2023-06-08T03:50:29.688 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:29.688 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:29.304383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-06-08T03:50:29.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f235c06ace0 msgr2=0x7f235c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f235c06ace0 0x7f235c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f237015c020 tx=0x7f2364007040 comp rx=0 tx=0).stop 2023-06-08T03:50:29.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 msgr2=0x7f2370157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 0x7f2370157fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23580057f0 tx=0x7f23600047c0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 shutdown_connections 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f237015c080 0x7f237015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f235c06ace0 0x7f235c06d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2358004850 0x7f23701e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 --2- 172.21.15.79:0/1239163670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2370156610 0x7f2370157fd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 >> 172.21.15.79:0/1239163670 conn(0x7f23700b1d20 msgr2=0x7f237015d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 shutdown_connections 2023-06-08T03:50:29.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.686+0000 7f23760d5700 1 -- 172.21.15.79:0/1239163670 wait complete. 2023-06-08T03:50:29.692 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 7 2023-06-08T03:50:29.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.820+0000 7f23c515b700 1 Processor -- start 2023-06-08T03:50:29.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.820+0000 7f23c515b700 1 -- start start 2023-06-08T03:50:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.820+0000 7f23c515b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c006b0d0 0x7f23c00ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.820+0000 7f23c515b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 0x7f23c00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23c515b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c00cea60 con 0x7f23c00b1650 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23c515b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00cf6e0 con 0x7f23c006b0d0 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23c515b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c00ced40 con 0x7f23c0160080 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 0x7f23c00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 0x7f23c00cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:48728/0 (socket says 172.21.15.79:48728) 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 -- 172.21.15.79:0/329644053 learned_addr learned my addr 172.21.15.79:0/329644053 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:29.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bed9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c006b0d0 0x7f23c00ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 -- 172.21.15.79:0/329644053 >> v2:172.21.15.156:3300/0 conn(0x7f23c006b0d0 msgr2=0x7f23c00ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 --2- 172.21.15.79:0/329644053 >> v2:172.21.15.156:3300/0 conn(0x7f23c006b0d0 0x7f23c00ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 -- 172.21.15.79:0/329644053 >> v1:172.21.15.156:6789/0 conn(0x7f23c00b1650 legacy=0x7f23c015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 -- 172.21.15.79:0/329644053 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00cf020 con 0x7f23c0160080 2023-06-08T03:50:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.821+0000 7f23bf59e700 1 --2- 172.21.15.79:0/329644053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 0x7f23c00cfcd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009e50 tx=0x7f23b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e324e42b896ff234 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23bdd9b700 1 -- 172.21.15.79:0/329644053 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b800e070 con 0x7f23c0160080 2023-06-08T03:50:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23bdd9b700 1 -- 172.21.15.79:0/329644053 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b800b910 con 0x7f23c0160080 2023-06-08T03:50:29.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23bdd9b700 1 -- 172.21.15.79:0/329644053 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b800bcc0 con 0x7f23c0160080 2023-06-08T03:50:29.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 -- 172.21.15.79:0/329644053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 msgr2=0x7f23c00cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 --2- 172.21.15.79:0/329644053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 0x7f23c00cfcd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009e50 tx=0x7f23b800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 -- 172.21.15.79:0/329644053 shutdown_connections 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 --2- 172.21.15.79:0/329644053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c0160080 0x7f23c00cfcd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 --2- 172.21.15.79:0/329644053 >> v2:172.21.15.156:3300/0 conn(0x7f23c006b0d0 0x7f23c00ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 -- 172.21.15.79:0/329644053 >> 172.21.15.79:0/329644053 conn(0x7f23c006a670 msgr2=0x7f23c0099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 -- 172.21.15.79:0/329644053 shutdown_connections 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.822+0000 7f23c515b700 1 -- 172.21.15.79:0/329644053 wait complete. 2023-06-08T03:50:29.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.824+0000 7f23c515b700 1 Processor -- start 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.824+0000 7f23c515b700 1 -- start start 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23c515b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c006b0d0 0x7f23c0155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23c515b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00b1650 0x7f23c00ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23c515b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 0x7f23c00c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23c515b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0157fb0 con 0x7f23c0160080 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23c515b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c00c5160 con 0x7f23c00b1650 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23c515b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c00b5cc0 con 0x7f23c006b0d0 2023-06-08T03:50:29.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 0x7f23c00c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 0x7f23c00c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34412/0 (socket says 172.21.15.79:34412) 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 -- 172.21.15.79:0/736822425 learned_addr learned my addr 172.21.15.79:0/736822425 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bed9d700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c006b0d0 0x7f23c0155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23be59c700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00b1650 0x7f23c00ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 -- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c006b0d0 msgr2=0x7f23c0155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c006b0d0 0x7f23c0155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 -- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00b1650 msgr2=0x7f23c00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00b1650 0x7f23c00ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 -- 172.21.15.79:0/736822425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8009b30 con 0x7f23c0160080 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.825+0000 7f23bf59e700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 0x7f23c00c4330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23b8012000 tx=0x7f23b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.826+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8004800 con 0x7f23c0160080 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.826+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b8004960 con 0x7f23c0160080 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.826+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c00c4e80 con 0x7f23c0160080 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.826+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b801b670 con 0x7f23c0160080 2023-06-08T03:50:29.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.826+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c0157bd0 con 0x7f23c0160080 2023-06-08T03:50:29.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.827+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c0073e90 con 0x7f23c0160080 2023-06-08T03:50:29.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.827+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23b801b7d0 con 0x7f23c0160080 2023-06-08T03:50:29.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.828+0000 7f23a77fe700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23ac06ad10 0x7f23ac06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:29.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.828+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f23b808aea0 con 0x7f23c0160080 2023-06-08T03:50:29.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.828+0000 7f23bed9d700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23ac06ad10 0x7f23ac06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:29.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.828+0000 7f23bed9d700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23ac06ad10 0x7f23ac06d1c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009be0 tx=0x7f23b0009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:29.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:29.832+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23b8055cf0 con 0x7f23c0160080 2023-06-08T03:50:30.026 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.022+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23c00c7590 con 0x7f23c0160080 2023-06-08T03:50:30.027 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.023+0000 7f23a77fe700 1 -- 172.21.15.79:0/736822425 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f23b8059310 con 0x7f23c0160080 2023-06-08T03:50:30.027 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:30.028 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23ac06ad10 msgr2=0x7f23ac06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23ac06ad10 0x7f23ac06d1c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009be0 tx=0x7f23b0009210 comp rx=0 tx=0).stop 2023-06-08T03:50:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 msgr2=0x7f23c00c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 0x7f23c00c4330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23b8012000 tx=0x7f23b8004ad0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 shutdown_connections 2023-06-08T03:50:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c0160080 0x7f23c00c4330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23ac06ad10 0x7f23ac06d1c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c006b0d0 0x7f23c0155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 --2- 172.21.15.79:0/736822425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00b1650 0x7f23c00ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 >> 172.21.15.79:0/736822425 conn(0x7f23c006a670 msgr2=0x7f23c006f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 shutdown_connections 2023-06-08T03:50:30.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.025+0000 7f23c515b700 1 -- 172.21.15.79:0/736822425 wait complete. 2023-06-08T03:50:30.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:30.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.156+0000 7f9f2e205700 1 Processor -- start 2023-06-08T03:50:30.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f2e205700 1 -- start start 2023-06-08T03:50:30.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f2e205700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f28158e30 0x7f9f280ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f2e205700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f28071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f2e205700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280caef0 con 0x7f9f28159200 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f2e205700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280cb1d0 con 0x7f9f28158e30 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f2e205700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280cb4b0 con 0x7f9f280cc490 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f26ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9f28159200 0x7f9f280cbd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57894/0 (socket says 172.21.15.79:57894) 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.157+0000 7f9f26ffd700 1 -- 172.21.15.79:0/3141457293 learned_addr learned my addr 172.21.15.79:0/3141457293 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f277fe700 1 --2- 172.21.15.79:0/3141457293 >> v2:172.21.15.156:3300/0 conn(0x7f9f28158e30 0x7f9f280ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f28071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297732730 0 0) 0x7f9f280caef0 con 0x7f9f28159200 2023-06-08T03:50:30.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f140035b0 con 0x7f9f28159200 2023-06-08T03:50:30.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 244520527 0 0) 0x7f9f140035b0 con 0x7f9f28159200 2023-06-08T03:50:30.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 msgr2=0x7f9f28071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f28071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 >> v2:172.21.15.156:3300/0 conn(0x7f9f28158e30 msgr2=0x7f9f280ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 --2- 172.21.15.79:0/3141457293 >> v2:172.21.15.156:3300/0 conn(0x7f9f28158e30 0x7f9f280ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280cb790 con 0x7f9f28159200 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f9f10001670 con 0x7f9f28159200 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f28071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f277fe700 1 --2- 172.21.15.79:0/3141457293 >> v2:172.21.15.156:3300/0 conn(0x7f9f28158e30 0x7f9f280ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 >> v1:172.21.15.79:6789/0 conn(0x7f9f28159200 legacy=0x7f9f280cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f14003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f14003fb0 0x7f9f14004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f14004d90 0x7f9f14005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280cb1d0 con 0x7f9f14004d90 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280cb4b0 con 0x7f9f14003fb0 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1400ba10 con 0x7f9f280cc490 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f14004d90 0x7f9f14005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f277fe700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f14003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f26ffd700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f14003fb0 0x7f9f14004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 -- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 msgr2=0x7f9f14003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f280cc490 0x7f9f14003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 -- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f14003fb0 msgr2=0x7f9f14004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f14003fb0 0x7f9f14004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.158+0000 7f9f27fff700 1 -- 172.21.15.79:0/3141457293 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20009a70 con 0x7f9f14004d90 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f27fff700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f14004d90 0x7f9f14005200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f28071000 tx=0x7f9f14005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04aa667c8be6626 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000edb0 con 0x7f9f14004d90 2023-06-08T03:50:30.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000f040 con 0x7f9f14004d90 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f267fc700 1 -- 172.21.15.79:0/3141457293 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20003c40 con 0x7f9f14004d90 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f2e205700 1 -- 172.21.15.79:0/3141457293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f14004d90 msgr2=0x7f9f14005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f2e205700 1 --2- 172.21.15.79:0/3141457293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f14004d90 0x7f9f14005200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f28071000 tx=0x7f9f14005880 comp rx=0 tx=0).stop 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f27fff700 1 -- 172.21.15.79:0/3141457293 reap_dead start 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f2e205700 1 -- 172.21.15.79:0/3141457293 shutdown_connections 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f2e205700 1 -- 172.21.15.79:0/3141457293 >> 172.21.15.79:0/3141457293 conn(0x7f9f280b1d20 msgr2=0x7f9f2806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:30.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f2e205700 1 -- 172.21.15.79:0/3141457293 shutdown_connections 2023-06-08T03:50:30.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.159+0000 7f9f2e205700 1 -- 172.21.15.79:0/3141457293 wait complete. 2023-06-08T03:50:30.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.161+0000 7f9f2e205700 1 Processor -- start 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.161+0000 7f9f2e205700 1 -- start start 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f2e205700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f2814cea0 0x7f9f2815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f2e205700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f280c66d0 0x7f9f280b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f2e205700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 0x7f9f280c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f2e205700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280b5200 con 0x7f9f280b4d90 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f2e205700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2815bfd0 con 0x7f9f280c66d0 2023-06-08T03:50:30.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f2e205700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2814d270 con 0x7f9f2814cea0 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 0x7f9f280c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 0x7f9f280c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34432/0 (socket says 172.21.15.79:34432) 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 -- 172.21.15.79:0/1658686164 learned_addr learned my addr 172.21.15.79:0/1658686164 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f277fe700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f2814cea0 0x7f9f2815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f26ffd700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f280c66d0 0x7f9f280b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 -- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f2814cea0 msgr2=0x7f9f2815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f2814cea0 0x7f9f2815b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 -- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f280c66d0 msgr2=0x7f9f280b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f280c66d0 0x7f9f280b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 -- 172.21.15.79:0/1658686164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2815ba10 con 0x7f9f280b4d90 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.162+0000 7f9f27fff700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 0x7f9f280c8440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9f20004000 tx=0x7f9f20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.163+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20026070 con 0x7f9f280b4d90 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.163+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f20004240 con 0x7f9f280b4d90 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.163+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2815bcf0 con 0x7f9f280b4d90 2023-06-08T03:50:30.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.163+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20007000 con 0x7f9f280b4d90 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.163+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f280b2720 con 0x7f9f280b4d90 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.164+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f28073e90 con 0x7f9f280b4d90 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.164+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f200043d0 con 0x7f9f280b4d90 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.164+0000 7f9f1ffff700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f0806ace0 0x7f9f0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.165+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9f20016070 con 0x7f9f280b4d90 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.165+0000 7f9f277fe700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f0806ace0 0x7f9f0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.165+0000 7f9f277fe700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f0806ace0 0x7f9f0806d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009e60 tx=0x7f9f18009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.169+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f20057740 con 0x7f9f280b4d90 2023-06-08T03:50:30.319 INFO:tasks.ceph.mds.a.smithi079.stderr:2023-06-08T03:50:30.315+0000 7f12daea6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-08T03:50:30.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.360+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f28073e90 con 0x7f9f280b4d90 2023-06-08T03:50:30.365 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.361+0000 7f9f1ffff700 1 -- 172.21.15.79:0/1658686164 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f9f2005ad60 con 0x7f9f280b4d90 2023-06-08T03:50:30.365 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:30.366 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:30.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f0806ace0 msgr2=0x7f9f0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f0806ace0 0x7f9f0806d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009e60 tx=0x7f9f18009210 comp rx=0 tx=0).stop 2023-06-08T03:50:30.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 msgr2=0x7f9f280c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 0x7f9f280c8440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9f20004000 tx=0x7f9f20009750 comp rx=0 tx=0).stop 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 shutdown_connections 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f280b4d90 0x7f9f280c8440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f0806ace0 0x7f9f0806d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f2814cea0 0x7f9f2815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 --2- 172.21.15.79:0/1658686164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f280c66d0 0x7f9f280b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 >> 172.21.15.79:0/1658686164 conn(0x7f9f280b1d20 msgr2=0x7f9f28069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:30.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 shutdown_connections 2023-06-08T03:50:30.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.363+0000 7f9f2e205700 1 -- 172.21.15.79:0/1658686164 wait complete. 2023-06-08T03:50:30.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.508+0000 7f86de16a700 1 Processor -- start 2023-06-08T03:50:30.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.508+0000 7f86de16a700 1 -- start start 2023-06-08T03:50:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86de16a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 0x7f86d8071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86de16a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80cee20 con 0x7f86d806a460 2023-06-08T03:50:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86de16a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d806fa00 con 0x7f86d815f6c0 2023-06-08T03:50:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86de16a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80cf330 con 0x7f86d80cd6c0 2023-06-08T03:50:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d6ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f86d806a460 0x7f86d8071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57912/0 (socket says 172.21.15.79:57912) 2023-06-08T03:50:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d6ffd700 1 -- 172.21.15.79:0/2601960213 learned_addr learned my addr 172.21.15.79:0/2601960213 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d7fff700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 0x7f86d8071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191629430 0 0) 0x7f86d80cee20 con 0x7f86d806a460 2023-06-08T03:50:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c00035b0 con 0x7f86d806a460 2023-06-08T03:50:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 773699518 0 0) 0x7f86c00035b0 con 0x7f86d806a460 2023-06-08T03:50:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 msgr2=0x7f86d8071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d67fc700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 0x7f86d8071760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.509+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 >> v1:172.21.15.156:6789/0 conn(0x7f86d815f6c0 legacy=0x7f86d8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80cd1e0 con 0x7f86d806a460 2023-06-08T03:50:30.517 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f86cc001670 con 0x7f86d806a460 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 >> v1:172.21.15.79:6789/0 conn(0x7f86d806a460 legacy=0x7f86d8071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 0x7f86c0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c0003fe0 0x7f86c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0004d90 0x7f86c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80cf330 con 0x7f86c0004d90 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c000bac0 con 0x7f86c0003fe0 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c000bc80 con 0x7f86d80cd6c0 2023-06-08T03:50:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0004d90 0x7f86c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d6ffd700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c0003fe0 0x7f86c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d77fe700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 0x7f86c0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 -- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 msgr2=0x7f86c0003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d80cd6c0 0x7f86c0003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 -- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c0003fe0 msgr2=0x7f86c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86c0003fe0 0x7f86c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 -- 172.21.15.79:0/2601960213 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00035b0 con 0x7f86c0004d90 2023-06-08T03:50:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d7fff700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0004d90 0x7f86c0005200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009b30 tx=0x7f86c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7534a504d1ab02ea server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000e6c0 con 0x7f86c0004d90 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d000e820 con 0x7f86c0004d90 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.510+0000 7f86d67fc700 1 -- 172.21.15.79:0/2601960213 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000ebb0 con 0x7f86c0004d90 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86de16a700 1 -- 172.21.15.79:0/2601960213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0004d90 msgr2=0x7f86c0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86de16a700 1 --2- 172.21.15.79:0/2601960213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86c0004d90 0x7f86c0005200 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009b30 tx=0x7f86c0005880 comp rx=0 tx=0).stop 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86d7fff700 1 -- 172.21.15.79:0/2601960213 reap_dead start 2023-06-08T03:50:30.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86de16a700 1 -- 172.21.15.79:0/2601960213 shutdown_connections 2023-06-08T03:50:30.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86de16a700 1 -- 172.21.15.79:0/2601960213 >> 172.21.15.79:0/2601960213 conn(0x7f86d80b1d20 msgr2=0x7f86d806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:30.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86de16a700 1 -- 172.21.15.79:0/2601960213 shutdown_connections 2023-06-08T03:50:30.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.511+0000 7f86de16a700 1 -- 172.21.15.79:0/2601960213 wait complete. 2023-06-08T03:50:30.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.512+0000 7f86de16a700 1 Processor -- start 2023-06-08T03:50:30.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.512+0000 7f86de16a700 1 -- start start 2023-06-08T03:50:30.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86de16a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d814ea00 0x7f86d814f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86de16a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d814edd0 0x7f86d80c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.523 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86de16a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 0x7f86d80c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86de16a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80c7830 con 0x7f86d80c6640 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86de16a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80c6bb0 con 0x7f86d814edd0 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86de16a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d80c6e90 con 0x7f86d814ea00 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 0x7f86d80c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 0x7f86d80c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34448/0 (socket says 172.21.15.79:34448) 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 -- 172.21.15.79:0/1170122684 learned_addr learned my addr 172.21.15.79:0/1170122684 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:30.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d6ffd700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d814edd0 0x7f86d80c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d77fe700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d814ea00 0x7f86d814f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 -- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d814ea00 msgr2=0x7f86d814f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d814ea00 0x7f86d814f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 -- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d814edd0 msgr2=0x7f86d80c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d814edd0 0x7f86d80c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 -- 172.21.15.79:0/1170122684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80c7170 con 0x7f86d80c6640 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.513+0000 7f86d7fff700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 0x7f86d80c9250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86d00090b0 tx=0x7f86d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.514+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d000e6c0 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.514+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d0013b10 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.514+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d80c7450 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.514+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d001c450 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.514+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d815c0c0 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.515+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d8073e90 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.515+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86d001c5b0 con 0x7f86d80c6640 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.515+0000 7f86bffff700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f86c406ace0 0x7f86c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.515+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f86d008ae60 con 0x7f86d80c6640 2023-06-08T03:50:30.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.516+0000 7f86d77fe700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f86c406ace0 0x7f86c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.516+0000 7f86d77fe700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f86c406ace0 0x7f86c406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86c8003c50 tx=0x7f86c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.520+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86d0055dd0 con 0x7f86d80c6640 2023-06-08T03:50:30.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.749+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86d8072370 con 0x7f86d80c6640 2023-06-08T03:50:30.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.749+0000 7f86bffff700 1 -- 172.21.15.79:0/1170122684 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f86d00593f0 con 0x7f86d80c6640 2023-06-08T03:50:30.754 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:30.754 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:29.304383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-06-08T03:50:30.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f86c406ace0 msgr2=0x7f86c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f86c406ace0 0x7f86c406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f86c8003c50 tx=0x7f86c8009040 comp rx=0 tx=0).stop 2023-06-08T03:50:30.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 msgr2=0x7f86d80c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 0x7f86d80c9250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86d00090b0 tx=0x7f86d0004750 comp rx=0 tx=0).stop 2023-06-08T03:50:30.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 shutdown_connections 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86d80c6640 0x7f86d80c9250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f86c406ace0 0x7f86c406d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86d814ea00 0x7f86d814f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 --2- 172.21.15.79:0/1170122684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86d814edd0 0x7f86d80c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 >> 172.21.15.79:0/1170122684 conn(0x7f86d80b1d20 msgr2=0x7f86d815fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 shutdown_connections 2023-06-08T03:50:30.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.752+0000 7f86de16a700 1 -- 172.21.15.79:0/1170122684 wait complete. 2023-06-08T03:50:30.758 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 7 2023-06-08T03:50:30.767 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:30.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.883+0000 7f8549681700 1 Processor -- start 2023-06-08T03:50:30.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.883+0000 7f8549681700 1 -- start start 2023-06-08T03:50:30.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8549681700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f8544164010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8549681700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854406ae60 con 0x7f85440ca630 2023-06-08T03:50:30.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8549681700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854406b9e0 con 0x7f8544159270 2023-06-08T03:50:30.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8549681700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85440cebc0 con 0x7f85440b16f0 2023-06-08T03:50:30.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f85427fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f85440ca630 0x7f85440b0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57928/0 (socket says 172.21.15.79:57928) 2023-06-08T03:50:30.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f85427fc700 1 -- 172.21.15.79:0/2622213344 learned_addr learned my addr 172.21.15.79:0/2622213344 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:30.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f85437fe700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f8544164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329076702 0 0) 0x7f854406ae60 con 0x7f85440ca630 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f852c0035b0 con 0x7f85440ca630 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010279007 0 0) 0x7f854406b9e0 con 0x7f8544159270 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f854406ae60 con 0x7f8544159270 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256578751 0 0) 0x7f852c0035b0 con 0x7f85440ca630 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 msgr2=0x7f8544164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f8544164010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 >> v1:172.21.15.156:6789/0 conn(0x7f8544159270 legacy=0x7f85440cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854406b0b0 con 0x7f85440ca630 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f8534001670 con 0x7f85440ca630 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f85437fe700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f8544164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.884+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 >> v1:172.21.15.79:6789/0 conn(0x7f85440ca630 legacy=0x7f85440b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c003c30 0x7f852c004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f852c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f852c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440cebc0 con 0x7f852c004850 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c00bb50 con 0x7f852c003c30 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c00bd10 con 0x7f85440b16f0 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f852c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8542ffd700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c003c30 0x7f852c004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85437fe700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f852c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 -- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 msgr2=0x7f852c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85440b16f0 0x7f852c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 -- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c003c30 msgr2=0x7f852c004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:30.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f852c003c30 0x7f852c004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 -- 172.21.15.79:0/2622213344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c009a70 con 0x7f852c004850 2023-06-08T03:50:30.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f85427fc700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f852c004cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85440caaa0 tx=0x7f852c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5de12ee55e49e3f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400cd20 con 0x7f852c004850 2023-06-08T03:50:30.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853400ce80 con 0x7f852c004850 2023-06-08T03:50:30.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.885+0000 7f8541ffb700 1 -- 172.21.15.79:0/2622213344 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534005ba0 con 0x7f852c004850 2023-06-08T03:50:30.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f8549681700 1 -- 172.21.15.79:0/2622213344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 msgr2=0x7f852c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f8549681700 1 --2- 172.21.15.79:0/2622213344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f852c004cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85440caaa0 tx=0x7f852c005880 comp rx=0 tx=0).stop 2023-06-08T03:50:30.899 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f85437fe700 1 -- 172.21.15.79:0/2622213344 reap_dead start 2023-06-08T03:50:30.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f8549681700 1 -- 172.21.15.79:0/2622213344 shutdown_connections 2023-06-08T03:50:30.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f8549681700 1 -- 172.21.15.79:0/2622213344 >> 172.21.15.79:0/2622213344 conn(0x7f85440cbfe0 msgr2=0x7f85440cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:30.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f8549681700 1 -- 172.21.15.79:0/2622213344 shutdown_connections 2023-06-08T03:50:30.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.886+0000 7f8549681700 1 -- 172.21.15.79:0/2622213344 wait complete. 2023-06-08T03:50:30.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.887+0000 7f8549681700 1 Processor -- start 2023-06-08T03:50:30.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.887+0000 7f8549681700 1 -- start start 2023-06-08T03:50:30.902 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8549681700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f85440c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8549681700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85441e16b0 0x7f85441e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8549681700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8544147760 0x7f8544147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8549681700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85441e2ea0 con 0x7f852c004850 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8549681700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8544148110 con 0x7f8544147760 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8549681700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85441e2220 con 0x7f85441e16b0 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f85440c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f85440c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34464/0 (socket says 172.21.15.79:34464) 2023-06-08T03:50:30.903 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 -- 172.21.15.79:0/3414293812 learned_addr learned my addr 172.21.15.79:0/3414293812 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f85427fc700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85441e16b0 0x7f85441e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f85437fe700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8544147760 0x7f8544147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 -- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85441e16b0 msgr2=0x7f85441e1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85441e16b0 0x7f85441e1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 -- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8544147760 msgr2=0x7f8544147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8544147760 0x7f8544147bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:30.904 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.888+0000 7f8542ffd700 1 -- 172.21.15.79:0/3414293812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85441e2500 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.889+0000 7f8542ffd700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f85440c8a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f853800f770 tx=0x7f853800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.889+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538005320 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.889+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85380054a0 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.889+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538006aa0 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.889+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85441e27e0 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.889+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8544154f20 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.890+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8544073e90 con 0x7f852c004850 2023-06-08T03:50:30.905 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.891+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8538006c00 con 0x7f852c004850 2023-06-08T03:50:30.906 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.891+0000 7f852b7fe700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f853006ad30 0x7f853006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:30.906 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.891+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8538088ec0 con 0x7f852c004850 2023-06-08T03:50:30.906 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.891+0000 7f85427fc700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f853006ad30 0x7f853006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:30.906 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.892+0000 7f85427fc700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f853006ad30 0x7f853006d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8534013040 tx=0x7f8534007f90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:30.906 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:30.895+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8538053df0 con 0x7f852c004850 2023-06-08T03:50:31.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.084+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85440b54e0 con 0x7f852c004850 2023-06-08T03:50:31.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.085+0000 7f852b7fe700 1 -- 172.21.15.79:0/3414293812 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f853801a030 con 0x7f852c004850 2023-06-08T03:50:31.090 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:31.090 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:31.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f853006ad30 msgr2=0x7f853006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f853006ad30 0x7f853006d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8534013040 tx=0x7f8534007f90 comp rx=0 tx=0).stop 2023-06-08T03:50:31.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 msgr2=0x7f85440c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f85440c8a70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f853800f770 tx=0x7f853800fa80 comp rx=0 tx=0).stop 2023-06-08T03:50:31.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 shutdown_connections 2023-06-08T03:50:31.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8544147760 0x7f8544147bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f852c004850 0x7f85440c8a70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f853006ad30 0x7f853006d1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 --2- 172.21.15.79:0/3414293812 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85441e16b0 0x7f85441e1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.087+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 >> 172.21.15.79:0/3414293812 conn(0x7f85440cbfe0 msgr2=0x7f85440aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.088+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 shutdown_connections 2023-06-08T03:50:31.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.088+0000 7f8549681700 1 -- 172.21.15.79:0/3414293812 wait complete. 2023-06-08T03:50:31.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-08T03:50:24.668853+0000', 'modified': '2023-06-08T03:50:29.304383+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6833/2931327716', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2931327716}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2931327716}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-06-08T03:50:31.506 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-08T03:50:31.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-08T03:50:31.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.623+0000 7f25354dc700 1 Processor -- start 2023-06-08T03:50:31.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.623+0000 7f25354dc700 1 -- start start 2023-06-08T03:50:31.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f25354dc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2530070ee0 0x7f253006d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f25354dc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25301604c0 con 0x7f253015c3f0 2023-06-08T03:50:31.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f25354dc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25300ce860 con 0x7f253015c020 2023-06-08T03:50:31.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f25354dc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25300ceb40 con 0x7f2530070ee0 2023-06-08T03:50:31.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252e7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f253015c3f0 0x7f25300707d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57950/0 (socket says 172.21.15.79:57950) 2023-06-08T03:50:31.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252e7fc700 1 -- 172.21.15.79:0/1432305748 learned_addr learned my addr 172.21.15.79:0/1432305748 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103045894 0 0) 0x7f25301604c0 con 0x7f253015c3f0 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25180035b0 con 0x7f253015c3f0 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1432305748 >> v2:172.21.15.156:3300/0 conn(0x7f2530070ee0 0x7f253006d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596531964 0 0) 0x7f25180035b0 con 0x7f253015c3f0 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 >> v2:172.21.15.156:3300/0 conn(0x7f2530070ee0 msgr2=0x7f253006d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 --2- 172.21.15.79:0/1432305748 >> v2:172.21.15.156:3300/0 conn(0x7f2530070ee0 0x7f253006d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 >> v1:172.21.15.156:6789/0 conn(0x7f253015c020 legacy=0x7f25300cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253015fe70 con 0x7f253015c3f0 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f2524001670 con 0x7f253015c3f0 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 >> v1:172.21.15.79:6789/0 conn(0x7f253015c3f0 legacy=0x7f25300707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2518003a00 0x7f2518003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2518004850 0x7f2518004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518005200 0x7f251800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300ceb40 con 0x7f2518003a00 2023-06-08T03:50:31.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251800c110 con 0x7f2518004850 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251800c2d0 con 0x7f2518005200 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252effd700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2518003a00 0x7f2518003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.624+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2518004850 0x7f2518004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518005200 0x7f251800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252effd700 1 -- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518005200 msgr2=0x7f251800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252effd700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518005200 0x7f251800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252effd700 1 -- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2518004850 msgr2=0x7f2518004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252effd700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2518004850 0x7f2518004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252effd700 1 -- 172.21.15.79:0/1432305748 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180035b0 con 0x7f2518003a00 2023-06-08T03:50:31.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518005200 0x7f251800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:31.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252f7fe700 1 -- 172.21.15.79:0/1432305748 reap_dead start 2023-06-08T03:50:31.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252effd700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2518003a00 0x7f2518003df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2520003eb0 tx=0x7f2518005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67630c5d11f6dc2d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520016070 con 0x7f2518003a00 2023-06-08T03:50:31.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520006860 con 0x7f2518003a00 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f252dffb700 1 -- 172.21.15.79:0/1432305748 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520003950 con 0x7f2518003a00 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 -- 172.21.15.79:0/1432305748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2518003a00 msgr2=0x7f2518003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2518003a00 0x7f2518003df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2520003eb0 tx=0x7f2518005880 comp rx=0 tx=0).stop 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 -- 172.21.15.79:0/1432305748 shutdown_connections 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 --2- 172.21.15.79:0/1432305748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2518003a00 0x7f2518003df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 -- 172.21.15.79:0/1432305748 >> 172.21.15.79:0/1432305748 conn(0x7f25300ced50 msgr2=0x7f25300cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 -- 172.21.15.79:0/1432305748 shutdown_connections 2023-06-08T03:50:31.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.625+0000 7f25354dc700 1 -- 172.21.15.79:0/1432305748 wait complete. 2023-06-08T03:50:31.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.627+0000 7f25354dc700 1 Processor -- start 2023-06-08T03:50:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.627+0000 7f25354dc700 1 -- start start 2023-06-08T03:50:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.627+0000 7f25354dc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518003a00 0x7f2530159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 0x7f25300c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25300c6370 0x7f25300c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300c3f60 con 0x7f25300c5e20 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300c4240 con 0x7f25300c6370 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25300c4520 con 0x7f2518003a00 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 0x7f25300c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 0x7f25300c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34486/0 (socket says 172.21.15.79:34486) 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 -- 172.21.15.79:0/1429593046 learned_addr learned my addr 172.21.15.79:0/1429593046 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252effd700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518003a00 0x7f2530159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25300c6370 0x7f25300c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 -- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518003a00 msgr2=0x7f2530159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518003a00 0x7f2530159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 -- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25300c6370 msgr2=0x7f25300c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25300c6370 0x7f25300c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 -- 172.21.15.79:0/1429593046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300c4800 con 0x7f25300c5e20 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25300c6370 0x7f25300c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252effd700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518003a00 0x7f2530159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 0x7f25300c5000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2524007290 tx=0x7f2524007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25177fe700 1 -- 172.21.15.79:0/1429593046 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240014e0 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25177fe700 1 -- 172.21.15.79:0/1429593046 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252400f8d0 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25300c4ae0 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25177fe700 1 -- 172.21.15.79:0/1429593046 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252400fc80 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.628+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25300c5540 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.630+0000 7f25177fe700 1 -- 172.21.15.79:0/1429593046 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f252401d460 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.630+0000 7f25177fe700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f251c06ace0 0x7f251c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.630+0000 7f25177fe700 1 -- 172.21.15.79:0/1429593046 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2524089860 con 0x7f25300c5e20 2023-06-08T03:50:31.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.630+0000 7f252effd700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f251c06ace0 0x7f251c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.631+0000 7f252effd700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f251c06ace0 0x7f251c06d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25300c8d70 tx=0x7f2520009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.632+0000 7f2541661b80 1 Processor -- start 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.632+0000 7f2541661b80 1 -- start start 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f2541661b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d846310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d84ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d893c70 0x555e8d8940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f2541661b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d7503b0 con 0x555e8d84a2f0 2023-06-08T03:50:31.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d751c60 con 0x555e8d893c70 2023-06-08T03:50:31.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e8d7a4570 con 0x555e8d84a6c0 2023-06-08T03:50:31.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d846310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d846310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34492/0 (socket says 172.21.15.79:34492) 2023-06-08T03:50:31.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 -- 172.21.15.79:0/1600574604 learned_addr learned my addr 172.21.15.79:0/1600574604 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d84ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d893c70 0x555e8d8940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 -- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 msgr2=0x555e8d84ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d84ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 -- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d893c70 msgr2=0x555e8d8940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.633+0000 7f252effd700 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d893c70 0x555e8d8940e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f252effd700 1 -- 172.21.15.79:0/1600574604 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e8d5cbd90 con 0x555e8d84a2f0 2023-06-08T03:50:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f252effd700 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d846310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f251800c0e0 tx=0x7f2520011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cb31aa95275022 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2514ff9700 1 -- 172.21.15.79:0/1600574604 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520016040 con 0x555e8d84a2f0 2023-06-08T03:50:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2514ff9700 1 -- 172.21.15.79:0/1600574604 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252002a070 con 0x555e8d84a2f0 2023-06-08T03:50:31.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2514ff9700 1 -- 172.21.15.79:0/1600574604 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520031680 con 0x555e8d84a2f0 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 -- 172.21.15.79:0/1600574604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 msgr2=0x555e8d846310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d846310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f251800c0e0 tx=0x7f2520011000 comp rx=0 tx=0).stop 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 -- 172.21.15.79:0/1600574604 shutdown_connections 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d893c70 0x555e8d8940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d846310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 --2- 172.21.15.79:0/1600574604 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d84ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.634+0000 7f2541661b80 1 -- 172.21.15.79:0/1600574604 >> 172.21.15.79:0/1600574604 conn(0x555e8d848ce0 msgr2=0x555e8d8914b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.635+0000 7f2541661b80 1 -- 172.21.15.79:0/1600574604 shutdown_connections 2023-06-08T03:50:31.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.635+0000 7f2541661b80 1 -- 172.21.15.79:0/1600574604 wait complete. 2023-06-08T03:50:31.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.635+0000 7f2541661b80 1 -- start start 2023-06-08T03:50:31.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.635+0000 7f2541661b80 1 Processor -- start 2023-06-08T03:50:31.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.636+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.636+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d88ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.636+0000 7f2541661b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 0x555e8d87a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.636+0000 7f2541661b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d79da00 con 0x555e8d893c70 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.636+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d73da70 con 0x555e8d84a2f0 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e8d73d770 con 0x555e8d84a6c0 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f2541661b80 10 client.0 _ll_register_callbacks cb 0x555e8d73b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 0x555e8d87a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 0x555e8d87a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34502/0 (socket says 172.21.15.79:34502) 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 -- 172.21.15.79:0/3809922067 learned_addr learned my addr 172.21.15.79:0/3809922067 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252effd700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252e7fc700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d88ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 -- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 msgr2=0x555e8d88ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d88ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 -- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 msgr2=0x555e8d8767a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8767a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 -- 172.21.15.79:0/3809922067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25180035b0 con 0x555e8d893c70 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f252f7fe700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 0x555e8d87a070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2530071c40 tx=0x7f2528007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acd2f723e99b8ee server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f250dffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f250dffb700 1 -- 172.21.15.79:0/3809922067 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528015600 con 0x555e8d893c70 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f250dffb700 1 -- 172.21.15.79:0/3809922067 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2528015760 con 0x555e8d893c70 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e8d83e850 con 0x555e8d893c70 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f250dffb700 1 -- 172.21.15.79:0/3809922067 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2528015920 con 0x555e8d893c70 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f2541661b80 10 client.4422 fetch_fsmap learned FSMap version 7 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x555e8d5b4c10 con 0x555e8d893c70 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.637+0000 7f250dffb700 1 -- 172.21.15.79:0/3809922067 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e8d83e850 con 0x555e8d893c70 2023-06-08T03:50:31.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f250dffb700 1 -- 172.21.15.79:0/3809922067 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f2528014070 con 0x555e8d893c70 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f2541661b80 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f2541661b80 10 client.4422 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f2541661b80 20 client.4422 populate_metadata read hostname 'smithi079' 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f2541661b80 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d86a170 0x555e8d8ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f2541661b80 4 client.4422 mds_command: new command op to 4260 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 --> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e8d7969e0 con 0x555e8d86a170 2023-06-08T03:50:31.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f252effd700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d86a170 0x555e8d8ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f250dffb700 10 client.4422 ms_handle_connect on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.638+0000 7f252effd700 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d86a170 0x555e8d8ae760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae3a45902e9d99e server_cookie=6b8f4eb96cb971f in_seq=0 out_seq=0 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f250dffb700 1 -- 172.21.15.79:0/3809922067 <== mds.0 v2:172.21.15.79:6832/2931327716 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e8d7969e0 con 0x555e8d86a170 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f250dffb700 10 client.4422 handle_command_reply: tid=0 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 client.4422 shutdown 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 msgr2=0x555e8d87a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 0x555e8d87a070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2530071c40 tx=0x7f2528007a70 comp rx=0 tx=0).stop 2023-06-08T03:50:31.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 shutdown_connections 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d893c70 0x555e8d87a070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8767a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d86a170 0x555e8d8ae760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 --2- 172.21.15.79:0/3809922067 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d84a6c0 0x555e8d88ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 >> 172.21.15.79:0/3809922067 conn(0x555e8d848ce0 msgr2=0x555e8d891420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f250dffb700 0 client.4422 ms_handle_reset on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 shutdown_connections 2023-06-08T03:50:31.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 1 -- 172.21.15.79:0/3809922067 wait complete. 2023-06-08T03:50:31.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.639+0000 7f2541661b80 20 client.4422 trim_cache size 0 max 16384 2023-06-08T03:50:31.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 Processor -- start 2023-06-08T03:50:31.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 -- start start 2023-06-08T03:50:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d84d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a6c0 0x555e8d8b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d88ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d6218d0 con 0x555e8d84a2f0 2023-06-08T03:50:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d84c630 con 0x555e8d84a6c0 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e8d891080 con 0x555e8d893c70 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f252effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d84d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f252effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d84d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34508/0 (socket says 172.21.15.79:34508) 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.657+0000 7f252effd700 1 -- 172.21.15.79:0/1495513719 learned_addr learned my addr 172.21.15.79:0/1495513719 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a6c0 0x555e8d8b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d88ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252effd700 1 -- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 msgr2=0x555e8d88ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252effd700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d88ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252effd700 1 -- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a6c0 msgr2=0x555e8d8b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252effd700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a6c0 0x555e8d8b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252effd700 1 -- 172.21.15.79:0/1495513719 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e8d5f2200 con 0x555e8d84a2f0 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252e7fc700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a6c0 0x555e8d8b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252f7fe700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d88ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:31.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f252effd700 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d84d730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2520014040 tx=0x7f2520007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae968b1980e786f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f25157fa700 1 -- 172.21.15.79:0/1495513719 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520037040 con 0x555e8d84a2f0 2023-06-08T03:50:31.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f25157fa700 1 -- 172.21.15.79:0/1495513719 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252001f070 con 0x555e8d84a2f0 2023-06-08T03:50:31.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f25157fa700 1 -- 172.21.15.79:0/1495513719 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520016040 con 0x555e8d84a2f0 2023-06-08T03:50:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f2541661b80 1 -- 172.21.15.79:0/1495513719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 msgr2=0x555e8d84d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f2541661b80 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d84d730 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2520014040 tx=0x7f2520007d50 comp rx=0 tx=0).stop 2023-06-08T03:50:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.658+0000 7f2541661b80 1 -- 172.21.15.79:0/1495513719 shutdown_connections 2023-06-08T03:50:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d88ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a2f0 0x555e8d84d730 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 --2- 172.21.15.79:0/1495513719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a6c0 0x555e8d8b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 -- 172.21.15.79:0/1495513719 >> 172.21.15.79:0/1495513719 conn(0x555e8d848ce0 msgr2=0x555e8d83fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 -- 172.21.15.79:0/1495513719 shutdown_connections 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 -- 172.21.15.79:0/1495513719 wait complete. 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 -- start start 2023-06-08T03:50:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.659+0000 7f2541661b80 1 Processor -- start 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 0x555e8d88f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d875610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d88f0b0 con 0x555e8d84a6c0 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e8d8b2c30 con 0x555e8d84a2f0 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e8d8b2ff0 con 0x555e8d893c70 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.660+0000 7f2541661b80 10 client.0 _ll_register_callbacks cb 0x555e8d73b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-08T03:50:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 0x555e8d88f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 0x555e8d88f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34514/0 (socket says 172.21.15.79:34514) 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 -- 172.21.15.79:0/2486802292 learned_addr learned my addr 172.21.15.79:0/2486802292 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252effd700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252f7fe700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d875610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 -- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 msgr2=0x555e8d875610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d875610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 -- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 msgr2=0x555e8d8a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 -- 172.21.15.79:0/2486802292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e8d5cbd90 con 0x555e8d84a6c0 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252f7fe700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d875610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f252e7fc700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 0x555e8d88f4d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f252400f4b0 tx=0x7f252405c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51dfa61b1ce53e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f250dffb700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f250dffb700 1 -- 172.21.15.79:0/2486802292 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252405e170 con 0x555e8d84a6c0 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f250dffb700 1 -- 172.21.15.79:0/2486802292 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f25240601a0 con 0x555e8d84a6c0 2023-06-08T03:50:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f252002a070 con 0x555e8d84a6c0 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f250dffb700 1 -- 172.21.15.79:0/2486802292 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2524060360 con 0x555e8d84a6c0 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 10 client.4428 fetch_fsmap learned FSMap version 7 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x555e8d5ca260 con 0x555e8d84a6c0 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f250dffb700 1 -- 172.21.15.79:0/2486802292 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252002a070 con 0x555e8d84a6c0 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f250dffb700 1 -- 172.21.15.79:0/2486802292 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f25240540d0 con 0x555e8d84a6c0 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 10 client.4428 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-08T03:50:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 20 client.4428 populate_metadata read hostname 'smithi079' 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d88fa60 0x555e8d875b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 4 client.4428 mds_command: new command op to 4260 tid=0[{"prefix": "status"}] 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.661+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 --> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e8d759000 con 0x555e8d88fa60 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f252effd700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d88fa60 0x555e8d875b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f250dffb700 10 client.4428 ms_handle_connect on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f252effd700 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d88fa60 0x555e8d875b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603a243c915b19f0 server_cookie=f5c1f25ec91a7aaf in_seq=0 out_seq=0 2023-06-08T03:50:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f250dffb700 1 -- 172.21.15.79:0/2486802292 <== mds.0 v2:172.21.15.79:6832/2931327716 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x555e8d759000 con 0x555e8d88fa60 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f250dffb700 10 client.4428 handle_command_reply: tid=0 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 client.4428 shutdown 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 msgr2=0x555e8d88f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 0x555e8d88f4d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f252400f4b0 tx=0x7f252405c320 comp rx=0 tx=0).stop 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 shutdown_connections 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x555e8d88fa60 0x555e8d875b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x555e8d893c70 0x555e8d875610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e8d84a2f0 0x555e8d8a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f250dffb700 0 client.4428 ms_handle_reset on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 --2- 172.21.15.79:0/2486802292 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555e8d84a6c0 0x555e8d88f4d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 >> 172.21.15.79:0/2486802292 conn(0x555e8d848ce0 msgr2=0x555e8d873520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 shutdown_connections 2023-06-08T03:50:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 1 -- 172.21.15.79:0/2486802292 wait complete. 2023-06-08T03:50:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.662+0000 7f2541661b80 20 client.4428 trim_cache size 0 max 16384 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "141881e0-5474-41d9-8205-1a44c31c5e0b", 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4260, 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2023-06-08T03:50:31.677 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2023-06-08T03:50:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 6.9931001620000002, 2023-06-08T03:50:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 7, 2023-06-08T03:50:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 25, 2023-06-08T03:50:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 24, 2023-06-08T03:50:31.678 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 12.381665922 2023-06-08T03:50:31.678 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:50:31.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f251c06ace0 msgr2=0x7f251c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f251c06ace0 0x7f251c06d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25300c8d70 tx=0x7f2520009030 comp rx=0 tx=0).stop 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 msgr2=0x7f25300c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 0x7f25300c5000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2524007290 tx=0x7f2524007650 comp rx=0 tx=0).stop 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 shutdown_connections 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25300c6370 0x7f25300c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f251c06ace0 0x7f251c06d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2518003a00 0x7f2530159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 --2- 172.21.15.79:0/1429593046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25300c5e20 0x7f25300c5000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 >> 172.21.15.79:0/1429593046 conn(0x7f25300ced50 msgr2=0x7f253006db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 shutdown_connections 2023-06-08T03:50:31.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.663+0000 7f25354dc700 1 -- 172.21.15.79:0/1429593046 wait complete. 2023-06-08T03:50:31.681 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-08T03:50:31.682 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:31.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-08T03:50:31.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:50:31.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:50:31.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.798+0000 7f969a680700 1 Processor -- start 2023-06-08T03:50:31.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.798+0000 7f969a680700 1 -- start start 2023-06-08T03:50:31.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f969a680700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f969415f6c0 0x7f9694069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f969a680700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96940cd6c0 0x7f969406a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f969a680700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96940cee20 con 0x7f969406f210 2023-06-08T03:50:31.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f969a680700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96940cf330 con 0x7f96940cd6c0 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f969a680700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969406fa00 con 0x7f969415f6c0 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f96937fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96940cd6c0 0x7f969406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f96937fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96940cd6c0 0x7f969406a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34520/0 (socket says 172.21.15.79:34520) 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f96937fe700 1 -- 172.21.15.79:0/4094731674 learned_addr learned my addr 172.21.15.79:0/4094731674 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.799+0000 7f9693fff700 1 --2- 172.21.15.79:0/4094731674 >> v2:172.21.15.156:3300/0 conn(0x7f969415f6c0 0x7f9694069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f96937fe700 1 -- 172.21.15.79:0/4094731674 >> v2:172.21.15.156:3300/0 conn(0x7f969415f6c0 msgr2=0x7f9694069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:31.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f96937fe700 1 --2- 172.21.15.79:0/4094731674 >> v2:172.21.15.156:3300/0 conn(0x7f969415f6c0 0x7f9694069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f96937fe700 1 -- 172.21.15.79:0/4094731674 >> v1:172.21.15.156:6789/0 conn(0x7f969406f210 legacy=0x7f96940cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f96937fe700 1 -- 172.21.15.79:0/4094731674 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96940cd1e0 con 0x7f96940cd6c0 2023-06-08T03:50:31.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f96937fe700 1 --2- 172.21.15.79:0/4094731674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96940cd6c0 0x7f969406a290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f969406be40 tx=0x7f9688009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef45c01fdb90693 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f9692ffd700 1 -- 172.21.15.79:0/4094731674 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688011830 con 0x7f96940cd6c0 2023-06-08T03:50:31.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f9692ffd700 1 -- 172.21.15.79:0/4094731674 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9688011990 con 0x7f96940cd6c0 2023-06-08T03:50:31.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.800+0000 7f9692ffd700 1 -- 172.21.15.79:0/4094731674 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688011d20 con 0x7f96940cd6c0 2023-06-08T03:50:31.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 -- 172.21.15.79:0/4094731674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96940cd6c0 msgr2=0x7f969406a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 --2- 172.21.15.79:0/4094731674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96940cd6c0 0x7f969406a290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f969406be40 tx=0x7f9688009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 -- 172.21.15.79:0/4094731674 shutdown_connections 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 --2- 172.21.15.79:0/4094731674 >> v2:172.21.15.156:3300/0 conn(0x7f969415f6c0 0x7f9694069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 --2- 172.21.15.79:0/4094731674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96940cd6c0 0x7f969406a290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 -- 172.21.15.79:0/4094731674 >> 172.21.15.79:0/4094731674 conn(0x7f96940b1d20 msgr2=0x7f969406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 -- 172.21.15.79:0/4094731674 shutdown_connections 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.801+0000 7f969a680700 1 -- 172.21.15.79:0/4094731674 wait complete. 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.802+0000 7f969a680700 1 Processor -- start 2023-06-08T03:50:31.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.802+0000 7f969a680700 1 -- start start 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f969a680700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f969406f210 0x7f96941e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f969a680700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96940cd6c0 0x7f96941f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f969a680700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 0x7f96941f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f969a680700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96941f6140 con 0x7f969415f6c0 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f969a680700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96941e7320 con 0x7f96940cd6c0 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f969a680700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96941e8030 con 0x7f969406f210 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f9698c1d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 0x7f96941f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f9698c1d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 0x7f96941f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34530/0 (socket says 172.21.15.79:34530) 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f9698c1d700 1 -- 172.21.15.79:0/3223153771 learned_addr learned my addr 172.21.15.79:0/3223153771 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f9693fff700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f969406f210 0x7f96941e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.803+0000 7f96937fe700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96940cd6c0 0x7f96941f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9698c1d700 1 -- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f969406f210 msgr2=0x7f96941e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9698c1d700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f969406f210 0x7f96941e7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9698c1d700 1 -- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96940cd6c0 msgr2=0x7f96941f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9698c1d700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96940cd6c0 0x7f96941f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:31.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9698c1d700 1 -- 172.21.15.79:0/3223153771 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009550 con 0x7f969415f6c0 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f96937fe700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96940cd6c0 0x7f96941f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9698c1d700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 0x7f96941f5c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f968c00cc80 tx=0x7f968c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c003c00 con 0x7f969415f6c0 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c003d60 con 0x7f969415f6c0 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96941e67a0 con 0x7f969415f6c0 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968c00aaa0 con 0x7f969415f6c0 2023-06-08T03:50:31.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.804+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96941e6d60 con 0x7f969415f6c0 2023-06-08T03:50:31.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.805+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96941f7460 con 0x7f969415f6c0 2023-06-08T03:50:31.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.806+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f968c00ac00 con 0x7f969415f6c0 2023-06-08T03:50:31.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.806+0000 7f9690ff9700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f968006ace0 0x7f968006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:31.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.806+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f968c08a030 con 0x7f969415f6c0 2023-06-08T03:50:31.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.806+0000 7f9693fff700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f968006ace0 0x7f968006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:31.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.807+0000 7f9693fff700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f968006ace0 0x7f968006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9684009b40 tx=0x7f9684009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:31.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:31.810+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f968c054e60 con 0x7f969415f6c0 2023-06-08T03:50:32.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.043+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96941f7460 con 0x7f969415f6c0 2023-06-08T03:50:32.047 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.043+0000 7f9690ff9700 1 -- 172.21.15.79:0/3223153771 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f96941f7460 con 0x7f969415f6c0 2023-06-08T03:50:32.048 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.045+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f968006ace0 msgr2=0x7f968006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f968006ace0 0x7f968006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9684009b40 tx=0x7f9684009210 comp rx=0 tx=0).stop 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 msgr2=0x7f96941f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 0x7f96941f5c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f968c00cc80 tx=0x7f968c00cf90 comp rx=0 tx=0).stop 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 shutdown_connections 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969415f6c0 0x7f96941f5c00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:32.050 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f968006ace0 0x7f968006d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:32.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f969406f210 0x7f96941e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:32.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 --2- 172.21.15.79:0/3223153771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96940cd6c0 0x7f96941f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:32.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 >> 172.21.15.79:0/3223153771 conn(0x7f96940b1d20 msgr2=0x7f9694069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:32.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 shutdown_connections 2023-06-08T03:50:32.051 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:32.046+0000 7f969a680700 1 -- 172.21.15.79:0/3223153771 wait complete. 2023-06-08T03:50:32.061 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-06-08T03:50:32.061 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-08T03:50:32.062 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-08T03:50:32.062 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-06-08T03:50:32.062 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:32.062 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-06-08T03:50:32.063 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-08T03:50:32.063 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-08T03:50:32.063 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-06-08T03:50:32.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.064 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-08T03:50:32.080 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:50:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:50:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:50:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-08T03:50:32.081 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:32.081 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:32.082 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-08T03:50:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42355sec preferred_lft 42355sec 2023-06-08T03:50:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::6d1d:ad0f:719a:51d1/64 scope link noprefixroute 2023-06-08T03:50:32.083 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:32.083 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-08T03:50:32.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.084 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:32.084 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:32.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2023-06-08T03:50:32.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2023-06-08T03:50:32.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2023-06-08T03:50:32.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-08T03:50:32.085 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:32.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.246 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-08T03:50:32.273 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-08T03:50:32.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.275 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-08T03:50:32.351 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-08T03:50:32.351 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-08T03:50:32.352 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-08T03:50:32.352 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-08T03:50:32.352 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-08T03:50:32.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.353 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:32.353 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:32.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-08T03:50:32.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-08T03:50:32.354 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-08T03:50:32.354 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:32.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.647 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-08T03:50:32.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.664 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2023-06-08T03:50:32.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.720 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:32.720 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:32.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-06-08T03:50:32.721 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-06-08T03:50:32.721 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:32.812 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-06-08T03:50:32.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:32.813 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:32.814 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:32.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-06-08T03:50:32.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-08T03:50:32.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-06-08T03:50:32.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-06-08T03:50:32.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-06-08T03:50:32.815 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:33.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:33.039 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:33.039 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:33.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2023-06-08T03:50:33.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2023-06-08T03:50:33.040 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:33.142 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-06-08T03:50:33.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:50:33.143 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:33.161 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-06-08T03:50:33.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:50:33.162 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:33.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:50:33.284 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:50:33.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:50:33.350 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:50:33.351 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:50:33.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:33.352 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:50:33.409 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-08T03:50:33.409 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-06-08T03:50:33.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:50:33.477 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:50:33.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:50:33.543 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:50:33.544 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:50:33.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:33.545 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:50:33.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:2023-06-08T03:50:33.543+0000 7f95b84275c0 -1 init, newargv = 0x555f32047320 newargc=15 2023-06-08T03:50:33.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[107289]: starting ceph client 2023-06-08T03:50:33.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[107289]: starting fuse 2023-06-08T03:50:33.564 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-08T03:50:33.564 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-06-08T03:50:33.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-08T03:50:33.565 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-08T03:50:33.565 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-08T03:50:33.565 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-08T03:50:33.566 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-08T03:50:33.566 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:33.566 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-08T03:50:33.567 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-06-08T03:50:33.567 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-08T03:50:33.567 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-06-08T03:50:33.567 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:33.568 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-08T03:50:33.568 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-08T03:50:33.568 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-08T03:50:33.569 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-08T03:50:33.569 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:33.570 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-08T03:50:33.570 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-08T03:50:33.570 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-08T03:50:33.571 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-08T03:50:33.571 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-08T03:50:33.571 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-08T03:50:33.571 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-08T03:50:33.572 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-08T03:50:33.572 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:33.572 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.admin")) 2023-06-08T03:50:33.573 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-08T03:50:33.686 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.admin.107289.asok 2023-06-08T03:50:33.868 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.107289.asok 2023-06-08T03:50:33.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:33.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.107289.asok status 2023-06-08T03:50:34.023 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:50:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-08T03:50:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "a02638f808b8fe11bccdd01222c0e6e2eb71120b", 2023-06-08T03:50:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)", 2023-06-08T03:50:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "admin", 2023-06-08T03:50:34.024 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "107289", 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4440, 2023-06-08T03:50:34.025 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 4440 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:50:34.026 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2875415473 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2875415473 2023-06-08T03:50:34.027 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.4440 192.168.0.1:0/2875415473", 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.1:0/2875415473", 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 7, 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 25, 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-08T03:50:34.028 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-08T03:50:34.029 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:50:34.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:34.034 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-08T03:50:34.034 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:34.050 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-08T03:50:34.051 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:34.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:34.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:34.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:34.118 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-06-08T03:50:34.209 INFO:teuthology.orchestra.run.smithi079.stdout:# file: . 2023-06-08T03:50:34.209 INFO:teuthology.orchestra.run.smithi079.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-06-08T03:50:34.209 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:34.210 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:34.210 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:50:34.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:34.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.382+0000 7fa15f198700 1 Processor -- start 2023-06-08T03:50:34.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.382+0000 7fa15f198700 1 -- start start 2023-06-08T03:50:34.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15f198700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa158159200 0x7fa1580cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15f198700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 0x7fa1580cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15f198700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580cacb0 con 0x7fa158158e30 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15f198700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1580caef0 con 0x7fa158159200 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15f198700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580cb1d0 con 0x7fa15815d730 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15cf34700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa158158e30 0x7fa1580ca5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58032/0 (socket says 172.21.15.79:58032) 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15cf34700 1 -- 172.21.15.79:0/3477425282 learned_addr learned my addr 172.21.15.79:0/3477425282 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa15d735700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 0x7fa1580cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa157fff700 1 --2- 172.21.15.79:0/3477425282 >> v2:172.21.15.156:3300/0 conn(0x7fa158159200 0x7fa1580cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291050567 0 0) 0x7fa14c001110 con 0x7fa158158e30 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1480035b0 con 0x7fa158158e30 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112228473 0 0) 0x7fa1480035b0 con 0x7fa158158e30 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 msgr2=0x7fa1580cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 0x7fa1580cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 >> v2:172.21.15.156:3300/0 conn(0x7fa158159200 msgr2=0x7fa1580cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 --2- 172.21.15.79:0/3477425282 >> v2:172.21.15.156:3300/0 conn(0x7fa158159200 0x7fa1580cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580cb4b0 con 0x7fa158158e30 2023-06-08T03:50:34.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.383+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fa14c001800 con 0x7fa158158e30 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 >> v1:172.21.15.79:6789/0 conn(0x7fa158158e30 legacy=0x7fa1580ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148003a70 0x7fa148003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 0x7fa148004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa148004d90 0x7fa148005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580caef0 con 0x7fa148004d90 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580cb1d0 con 0x7fa148003a70 2023-06-08T03:50:34.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14800ba10 con 0x7fa15815d730 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa148004d90 0x7fa148005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15cf34700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148003a70 0x7fa148003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa157fff700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 0x7fa148004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 -- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 msgr2=0x7fa148004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa15815d730 0x7fa148004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 -- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148003a70 msgr2=0x7fa148003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa148003a70 0x7fa148003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 -- 172.21.15.79:0/3477425282 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480035b0 con 0x7fa148004d90 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa15d735700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa148004d90 0x7fa148005200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa15000e170 tx=0x7fa148005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73a676b59d1b74e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15000ec30 con 0x7fa148004d90 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15000f040 con 0x7fa148004d90 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.384+0000 7fa1577fe700 1 -- 172.21.15.79:0/3477425282 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150014480 con 0x7fa148004d90 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15f198700 1 -- 172.21.15.79:0/3477425282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa148004d90 msgr2=0x7fa148005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15f198700 1 --2- 172.21.15.79:0/3477425282 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa148004d90 0x7fa148005200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa15000e170 tx=0x7fa148005880 comp rx=0 tx=0).stop 2023-06-08T03:50:34.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15d735700 1 -- 172.21.15.79:0/3477425282 reap_dead start 2023-06-08T03:50:34.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15f198700 1 -- 172.21.15.79:0/3477425282 shutdown_connections 2023-06-08T03:50:34.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15f198700 1 -- 172.21.15.79:0/3477425282 >> 172.21.15.79:0/3477425282 conn(0x7fa1580b1d20 msgr2=0x7fa15806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:34.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15f198700 1 -- 172.21.15.79:0/3477425282 shutdown_connections 2023-06-08T03:50:34.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.385+0000 7fa15f198700 1 -- 172.21.15.79:0/3477425282 wait complete. 2023-06-08T03:50:34.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.386+0000 7fa15f198700 1 Processor -- start 2023-06-08T03:50:34.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.386+0000 7fa15f198700 1 -- start start 2023-06-08T03:50:34.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.387+0000 7fa15f198700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa148004d90 0x7fa158157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.387+0000 7fa15f198700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 0x7fa1580cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.387+0000 7fa15f198700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa158149080 0x7fa158149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.387+0000 7fa15f198700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ce2a0 con 0x7fa1580cd8f0 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.387+0000 7fa15f198700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1581548c0 con 0x7fa158149080 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.387+0000 7fa15f198700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1581e4e10 con 0x7fa148004d90 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 0x7fa1580cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 0x7fa1580cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34566/0 (socket says 172.21.15.79:34566) 2023-06-08T03:50:34.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 -- 172.21.15.79:0/2273564693 learned_addr learned my addr 172.21.15.79:0/2273564693 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa15cf34700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa148004d90 0x7fa158157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa15d735700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa158149080 0x7fa158149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 -- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa148004d90 msgr2=0x7fa158157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa148004d90 0x7fa158157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 -- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa158149080 msgr2=0x7fa158149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa158149080 0x7fa158149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 -- 172.21.15.79:0/2273564693 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1581e50f0 con 0x7fa1580cd8f0 2023-06-08T03:50:34.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa157fff700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 0x7fa1580cdd60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa144011a30 tx=0x7fa144011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144011ed0 con 0x7fa1580cd8f0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144004240 con 0x7fa1580cd8f0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f730 con 0x7fa1580cd8f0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1581e53d0 con 0x7fa1580cd8f0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.388+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1581e4ac0 con 0x7fa1580cd8f0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.389+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1580ca5a0 con 0x7fa1580cd8f0 2023-06-08T03:50:34.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.390+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa14400f890 con 0x7fa1580cd8f0 2023-06-08T03:50:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.390+0000 7fa1557fa700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa13c06ace0 0x7fa13c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.390+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa144089d60 con 0x7fa1580cd8f0 2023-06-08T03:50:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.390+0000 7fa15cf34700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa13c06ace0 0x7fa13c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.390+0000 7fa15cf34700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa13c06ace0 0x7fa13c06d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa14c001e80 tx=0x7fa14c007b00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:34.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.394+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa144054a80 con 0x7fa1580cd8f0 2023-06-08T03:50:34.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.584+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa158154a30 con 0x7fa1580cd8f0 2023-06-08T03:50:34.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.585+0000 7fa1557fa700 1 -- 172.21.15.79:0/2273564693 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fa144019090 con 0x7fa1580cd8f0 2023-06-08T03:50:34.589 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:34.590 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa13c06ace0 msgr2=0x7fa13c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa13c06ace0 0x7fa13c06d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa14c001e80 tx=0x7fa14c007b00 comp rx=0 tx=0).stop 2023-06-08T03:50:34.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 msgr2=0x7fa1580cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 0x7fa1580cdd60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa144011a30 tx=0x7fa144011d40 comp rx=0 tx=0).stop 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 shutdown_connections 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa158149080 0x7fa158149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa13c06ace0 0x7fa13c06d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa148004d90 0x7fa158157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 --2- 172.21.15.79:0/2273564693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1580cd8f0 0x7fa1580cdd60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 >> 172.21.15.79:0/2273564693 conn(0x7fa1580b1d20 msgr2=0x7fa1580cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 shutdown_connections 2023-06-08T03:50:34.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.587+0000 7fa15f198700 1 -- 172.21.15.79:0/2273564693 wait complete. 2023-06-08T03:50:34.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:50:34.604 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:50:34.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.722+0000 7f794cee6700 1 Processor -- start 2023-06-08T03:50:34.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.722+0000 7f794cee6700 1 -- start start 2023-06-08T03:50:34.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.723+0000 7f794cee6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f794815fb10 0x7f7948070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.723+0000 7f794cee6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7948070bd0 con 0x7f79480cb1e0 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.723+0000 7f794cee6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79480cad50 con 0x7f79480cce80 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.723+0000 7f794cee6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79480ce570 con 0x7f794815fb10 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.723+0000 7f7946d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f79480cb1e0 0x7f79480cb660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58052/0 (socket says 172.21.15.79:58052) 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.723+0000 7f7946d9d700 1 -- 172.21.15.79:0/1964765900 learned_addr learned my addr 172.21.15.79:0/1964765900 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f7945d9b700 1 --2- 172.21.15.79:0/1964765900 >> v2:172.21.15.156:3300/0 conn(0x7f794815fb10 0x7f7948070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245102150 0 0) 0x7f7940001490 con 0x7f79480cb1e0 2023-06-08T03:50:34.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79300035b0 con 0x7f79480cb1e0 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636887571 0 0) 0x7f79480cad50 con 0x7f79480cce80 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7940001490 con 0x7f79480cce80 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101945029 0 0) 0x7f79300035b0 con 0x7f79480cb1e0 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f7945d9b700 1 -- 172.21.15.79:0/1964765900 >> v1:172.21.15.156:6789/0 conn(0x7f79480cce80 legacy=0x7f79480cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f7945d9b700 1 -- 172.21.15.79:0/1964765900 >> v1:172.21.15.79:6789/0 conn(0x7f79480cb1e0 legacy=0x7f79480cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f7945d9b700 1 -- 172.21.15.79:0/1964765900 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7948160080 con 0x7f794815fb10 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f7945d9b700 1 --2- 172.21.15.79:0/1964765900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f794815fb10 0x7f7948070690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7938009840 tx=0x7f7938009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21521345a196c811 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:34.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940001b80 con 0x7f794815fb10 2023-06-08T03:50:34.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938004030 con 0x7f794815fb10 2023-06-08T03:50:34.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.724+0000 7f794559a700 1 -- 172.21.15.79:0/1964765900 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380043c0 con 0x7f794815fb10 2023-06-08T03:50:34.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 -- 172.21.15.79:0/1964765900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f794815fb10 msgr2=0x7f7948070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 --2- 172.21.15.79:0/1964765900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f794815fb10 0x7f7948070690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7938009840 tx=0x7f7938009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:34.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 -- 172.21.15.79:0/1964765900 shutdown_connections 2023-06-08T03:50:34.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 --2- 172.21.15.79:0/1964765900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f794815fb10 0x7f7948070690 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 -- 172.21.15.79:0/1964765900 >> 172.21.15.79:0/1964765900 conn(0x7f794806e1e0 msgr2=0x7f794806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 -- 172.21.15.79:0/1964765900 shutdown_connections 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.725+0000 7f794cee6700 1 -- 172.21.15.79:0/1964765900 wait complete. 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.726+0000 7f794cee6700 1 Processor -- start 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.726+0000 7f794cee6700 1 -- start start 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794cee6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 0x7f7948152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794cee6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79480cce80 0x7f794815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794cee6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f794815fb10 0x7f79480b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794cee6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794815dbf0 con 0x7f79480cb1e0 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794cee6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794815ded0 con 0x7f79480cce80 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794cee6700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794815e1b0 con 0x7f794815fb10 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 0x7f7948152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 0x7f7948152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34578/0 (socket says 172.21.15.79:34578) 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 -- 172.21.15.79:0/2853662012 learned_addr learned my addr 172.21.15.79:0/2853662012 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:34.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f7945d9b700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79480cce80 0x7f794815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f7946d9d700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f794815fb10 0x7f79480b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 -- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f794815fb10 msgr2=0x7f79480b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f794815fb10 0x7f79480b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 -- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79480cce80 msgr2=0x7f794815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79480cce80 0x7f794815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f794659c700 1 -- 172.21.15.79:0/2853662012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380094a0 con 0x7f79480cb1e0 2023-06-08T03:50:34.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.727+0000 7f7946d9d700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f794815fb10 0x7f79480b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.728+0000 7f794659c700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 0x7f7948152ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c500 tx=0x7f793c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.728+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0052d0 con 0x7f79480cb1e0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.728+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c005450 con 0x7f79480cb1e0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.728+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79481593c0 con 0x7f79480cb1e0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.728+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0038c0 con 0x7f79480cb1e0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.728+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79481530e0 con 0x7f79480cb1e0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.729+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7948159220 con 0x7f79480cb1e0 2023-06-08T03:50:34.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.729+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f793c003a20 con 0x7f79480cb1e0 2023-06-08T03:50:34.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.730+0000 7f7936ffd700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f792806ace0 0x7f792806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:34.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.730+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f793c088bd0 con 0x7f79480cb1e0 2023-06-08T03:50:34.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.730+0000 7f7945d9b700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f792806ace0 0x7f792806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:34.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.730+0000 7f7945d9b700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f792806ace0 0x7f792806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7938009840 tx=0x7f7938008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:34.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.734+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7948159220 con 0x7f79480cb1e0 2023-06-08T03:50:34.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.921+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7948159220 con 0x7f79480cb1e0 2023-06-08T03:50:34.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.921+0000 7f7936ffd700 1 -- 172.21.15.79:0/2853662012 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f793c017020 con 0x7f79480cb1e0 2023-06-08T03:50:34.925 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:50:34.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f792806ace0 msgr2=0x7f792806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f792806ace0 0x7f792806d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7938009840 tx=0x7f7938008030 comp rx=0 tx=0).stop 2023-06-08T03:50:34.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 msgr2=0x7f7948152ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:34.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 0x7f7948152ba0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c500 tx=0x7f793c00c810 comp rx=0 tx=0).stop 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 shutdown_connections 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f792806ace0 0x7f792806d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f794815fb10 0x7f79480b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f79480cb1e0 0x7f7948152ba0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 --2- 172.21.15.79:0/2853662012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79480cce80 0x7f794815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 >> 172.21.15.79:0/2853662012 conn(0x7f794806e1e0 msgr2=0x7f794815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:34.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 shutdown_connections 2023-06-08T03:50:34.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:34.923+0000 7f794cee6700 1 -- 172.21.15.79:0/2853662012 wait complete. 2023-06-08T03:50:34.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:50:35.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.057+0000 7fcd4d0d0700 1 Processor -- start 2023-06-08T03:50:35.061 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.057+0000 7fcd4d0d0700 1 -- start start 2023-06-08T03:50:35.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4d0d0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd4806d8a0 0x7fcd48071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4d0d0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcd48069d50 0x7fcd4806a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4d0d0700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd480cd140 con 0x7fcd4815c1f0 2023-06-08T03:50:35.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4d0d0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4806a730 con 0x7fcd48069d50 2023-06-08T03:50:35.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4d0d0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd48071110 con 0x7fcd4806d8a0 2023-06-08T03:50:35.063 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcd48069d50 0x7fcd4806a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcd48069d50 0x7fcd4806a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34590/0 (socket says 172.21.15.79:34590) 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 -- 172.21.15.79:0/2786936001 learned_addr learned my addr 172.21.15.79:0/2786936001 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4659c700 1 --2- 172.21.15.79:0/2786936001 >> v2:172.21.15.156:3300/0 conn(0x7fcd4806d8a0 0x7fcd48071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 -- 172.21.15.79:0/2786936001 >> v2:172.21.15.156:3300/0 conn(0x7fcd4806d8a0 msgr2=0x7fcd48071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 --2- 172.21.15.79:0/2786936001 >> v2:172.21.15.156:3300/0 conn(0x7fcd4806d8a0 0x7fcd48071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 -- 172.21.15.79:0/2786936001 >> v1:172.21.15.156:6789/0 conn(0x7fcd4815c1f0 legacy=0x7fcd480cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 -- 172.21.15.79:0/2786936001 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd48071be0 con 0x7fcd48069d50 2023-06-08T03:50:35.064 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd4759e700 1 --2- 172.21.15.79:0/2786936001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd48069d50 0x7fcd4806a1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd480d0670 tx=0x7fcd4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35110fc592015952 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd45d9b700 1 -- 172.21.15.79:0/2786936001 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000e070 con 0x7fcd48069d50 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd45d9b700 1 -- 172.21.15.79:0/2786936001 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40003a20 con 0x7fcd48069d50 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.058+0000 7fcd45d9b700 1 -- 172.21.15.79:0/2786936001 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003dd0 con 0x7fcd48069d50 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/2786936001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd48069d50 msgr2=0x7fcd4806a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/2786936001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd48069d50 0x7fcd4806a1f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd480d0670 tx=0x7fcd4000b330 comp rx=0 tx=0).stop 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/2786936001 shutdown_connections 2023-06-08T03:50:35.065 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/2786936001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd48069d50 0x7fcd4806a1f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/2786936001 >> v2:172.21.15.156:3300/0 conn(0x7fcd4806d8a0 0x7fcd48071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/2786936001 >> 172.21.15.79:0/2786936001 conn(0x7fcd480b1d30 msgr2=0x7fcd4806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:35.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/2786936001 shutdown_connections 2023-06-08T03:50:35.066 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.059+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/2786936001 wait complete. 2023-06-08T03:50:35.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.061+0000 7fcd4d0d0700 1 Processor -- start 2023-06-08T03:50:35.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.061+0000 7fcd4d0d0700 1 -- start start 2023-06-08T03:50:35.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4d0d0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd48069d50 0x7fcd4814a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4d0d0700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd4806d8a0 0x7fcd480c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4d0d0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 0x7fcd48149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4d0d0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48147740 con 0x7fcd4815c1f0 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4d0d0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48147a20 con 0x7fcd48069d50 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4d0d0700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd48147d00 con 0x7fcd4806d8a0 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 0x7fcd48149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 0x7fcd48149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34596/0 (socket says 172.21.15.79:34596) 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 -- 172.21.15.79:0/1467386734 learned_addr learned my addr 172.21.15.79:0/1467386734 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4659c700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd4806d8a0 0x7fcd480c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd46d9d700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd48069d50 0x7fcd4814a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 -- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd4806d8a0 msgr2=0x7fcd480c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd4806d8a0 0x7fcd480c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 -- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd48069d50 msgr2=0x7fcd4814a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd48069d50 0x7fcd4814a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 -- 172.21.15.79:0/1467386734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40009b30 con 0x7fcd4815c1f0 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.062+0000 7fcd4759e700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 0x7fcd48149630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd40015040 tx=0x7fcd40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.063+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd400094d0 con 0x7fcd4815c1f0 2023-06-08T03:50:35.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.063+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40004240 con 0x7fcd4815c1f0 2023-06-08T03:50:35.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.063+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004630 con 0x7fcd4815c1f0 2023-06-08T03:50:35.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.063+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd48147fe0 con 0x7fcd4815c1f0 2023-06-08T03:50:35.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.063+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4815a810 con 0x7fcd4815c1f0 2023-06-08T03:50:35.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.063+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4815aa00 con 0x7fcd4815c1f0 2023-06-08T03:50:35.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.064+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd40019070 con 0x7fcd4815c1f0 2023-06-08T03:50:35.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.064+0000 7fcd2f7fe700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcd3406ace0 0x7fcd3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.064+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcd4008a550 con 0x7fcd4815c1f0 2023-06-08T03:50:35.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.065+0000 7fcd46d9d700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcd3406ace0 0x7fcd3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.065+0000 7fcd46d9d700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcd3406ace0 0x7fcd3406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009960 tx=0x7fcd3c0090e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.069+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcd40058e30 con 0x7fcd4815c1f0 2023-06-08T03:50:35.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.255+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd48147880 con 0x7fcd4815c1f0 2023-06-08T03:50:35.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.255+0000 7fcd2f7fe700 1 -- 172.21.15.79:0/1467386734 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd48147880 con 0x7fcd4815c1f0 2023-06-08T03:50:35.259 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:50:35.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcd3406ace0 msgr2=0x7fcd3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcd3406ace0 0x7fcd3406d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009960 tx=0x7fcd3c0090e0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 msgr2=0x7fcd48149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 0x7fcd48149630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcd40015040 tx=0x7fcd40004ad0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 shutdown_connections 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd4815c1f0 0x7fcd48149630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd48069d50 0x7fcd4814a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcd3406ace0 0x7fcd3406d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 --2- 172.21.15.79:0/1467386734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd4806d8a0 0x7fcd480c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 >> 172.21.15.79:0/1467386734 conn(0x7fcd480b1d30 msgr2=0x7fcd480b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:35.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 shutdown_connections 2023-06-08T03:50:35.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.257+0000 7fcd4d0d0700 1 -- 172.21.15.79:0/1467386734 wait complete. 2023-06-08T03:50:35.273 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:50:35.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.391+0000 7ff23e19e700 1 Processor -- start 2023-06-08T03:50:35.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.391+0000 7ff23e19e700 1 -- start start 2023-06-08T03:50:35.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23e19e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff23806a460 0x7ff2380cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23e19e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23815c750 con 0x7ff23815c380 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23e19e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2380b11d0 con 0x7ff238160860 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23e19e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23806e0a0 con 0x7ff23806a460 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23c99b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff23806a460 0x7ff2380cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23c99b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff23806a460 0x7ff2380cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34602/0 (socket says 172.21.15.79:34602) 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23c99b700 1 -- 172.21.15.79:0/3775221108 learned_addr learned my addr 172.21.15.79:0/3775221108 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:35.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23d19c700 1 --1- 172.21.15.79:0/3775221108 >> v1:172.21.15.79:6789/0 conn(0x7ff23815c380 0x7ff238069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58076/0 (socket says 172.21.15.79:58076) 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23c99b700 1 -- 172.21.15.79:0/3775221108 >> v1:172.21.15.156:6789/0 conn(0x7ff238160860 legacy=0x7ff2380712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23c99b700 1 -- 172.21.15.79:0/3775221108 >> v1:172.21.15.79:6789/0 conn(0x7ff23815c380 legacy=0x7ff238069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff23c99b700 1 -- 172.21.15.79:0/3775221108 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2380b1470 con 0x7ff23806a460 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.392+0000 7ff22ffff700 1 -- 172.21.15.79:0/3775221108 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661698749 0 0) 0x7ff23815c750 con 0x7ff23815c380 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23c99b700 1 --2- 172.21.15.79:0/3775221108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff23806a460 0x7ff2380cd1f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098f0 tx=0x7ff234009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae77ff09d6359db server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff22ffff700 1 -- 172.21.15.79:0/3775221108 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400b8a0 con 0x7ff23806a460 2023-06-08T03:50:35.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff22ffff700 1 -- 172.21.15.79:0/3775221108 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23400ba00 con 0x7ff23806a460 2023-06-08T03:50:35.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff22ffff700 1 -- 172.21.15.79:0/3775221108 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400bd90 con 0x7ff23806a460 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 -- 172.21.15.79:0/3775221108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff23806a460 msgr2=0x7ff2380cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3775221108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff23806a460 0x7ff2380cd1f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098f0 tx=0x7ff234009c00 comp rx=0 tx=0).stop 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 -- 172.21.15.79:0/3775221108 shutdown_connections 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3775221108 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff23806a460 0x7ff2380cd1f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 -- 172.21.15.79:0/3775221108 >> 172.21.15.79:0/3775221108 conn(0x7ff23806ba90 msgr2=0x7ff23806d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 -- 172.21.15.79:0/3775221108 shutdown_connections 2023-06-08T03:50:35.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.393+0000 7ff23e19e700 1 -- 172.21.15.79:0/3775221108 wait complete. 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 Processor -- start 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 -- start start 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff23815c380 0x7ff2381536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff238160860 0x7ff238147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 0x7ff2380b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380b3110 con 0x7ff2381479c0 2023-06-08T03:50:35.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380b33f0 con 0x7ff238160860 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.395+0000 7ff23e19e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380b36d0 con 0x7ff23815c380 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 0x7ff2380b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 0x7ff2380b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34610/0 (socket says 172.21.15.79:34610) 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 -- 172.21.15.79:0/3485339668 learned_addr learned my addr 172.21.15.79:0/3485339668 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d19c700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff23815c380 0x7ff2381536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23c99b700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff238160860 0x7ff238147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 -- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff23815c380 msgr2=0x7ff2381536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff23815c380 0x7ff2381536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 -- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff238160860 msgr2=0x7ff238147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff238160860 0x7ff238147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 -- 172.21.15.79:0/3485339668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234009550 con 0x7ff2381479c0 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23d99d700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 0x7ff2380b2450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff23000cb50 tx=0x7ff23000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230003c00 con 0x7ff2381479c0 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230003d60 con 0x7ff2381479c0 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000ab70 con 0x7ff2381479c0 2023-06-08T03:50:35.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2380b39b0 con 0x7ff2381479c0 2023-06-08T03:50:35.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.396+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff238147e40 con 0x7ff2381479c0 2023-06-08T03:50:35.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.397+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2380b3ed0 con 0x7ff2381479c0 2023-06-08T03:50:35.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.398+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff23000acd0 con 0x7ff2381479c0 2023-06-08T03:50:35.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.398+0000 7ff22dffb700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff21c06ace0 0x7ff21c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.398+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff230089810 con 0x7ff2381479c0 2023-06-08T03:50:35.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.398+0000 7ff23d19c700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff21c06ace0 0x7ff21c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.399+0000 7ff23d19c700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff21c06ace0 0x7ff21c06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff228007fd0 tx=0x7ff228007c20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.402+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff2380b3ed0 con 0x7ff2381479c0 2023-06-08T03:50:35.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.590+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2380b3ed0 con 0x7ff2381479c0 2023-06-08T03:50:35.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.590+0000 7ff22dffb700 1 -- 172.21.15.79:0/3485339668 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2380b3ed0 con 0x7ff2381479c0 2023-06-08T03:50:35.595 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:35.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff21c06ace0 msgr2=0x7ff21c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff21c06ace0 0x7ff21c06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff228007fd0 tx=0x7ff228007c20 comp rx=0 tx=0).stop 2023-06-08T03:50:35.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 msgr2=0x7ff2380b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 0x7ff2380b2450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff23000cb50 tx=0x7ff23000cf10 comp rx=0 tx=0).stop 2023-06-08T03:50:35.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 shutdown_connections 2023-06-08T03:50:35.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2381479c0 0x7ff2380b2450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff21c06ace0 0x7ff21c06d190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff23815c380 0x7ff2381536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 --2- 172.21.15.79:0/3485339668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff238160860 0x7ff238147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 >> 172.21.15.79:0/3485339668 conn(0x7ff23806a460 msgr2=0x7ff2380cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:35.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 shutdown_connections 2023-06-08T03:50:35.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.592+0000 7ff23e19e700 1 -- 172.21.15.79:0/3485339668 wait complete. 2023-06-08T03:50:35.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:50:35.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f969574a700 1 Processor -- start 2023-06-08T03:50:35.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f969574a700 1 -- start start 2023-06-08T03:50:35.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f969574a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96900715a0 0x7f969015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f969574a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969006db90 con 0x7f969006c050 2023-06-08T03:50:35.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f969574a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96900ca400 con 0x7f96900ca890 2023-06-08T03:50:35.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f969574a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969015e920 con 0x7f96900715a0 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f968effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96900715a0 0x7f969015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f968effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f96900715a0 0x7f969015e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34614/0 (socket says 172.21.15.79:34614) 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f968e7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f969006c050 0x7f9690071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58092/0 (socket says 172.21.15.79:58092) 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.727+0000 7f968effd700 1 -- 172.21.15.79:0/4173410929 learned_addr learned my addr 172.21.15.79:0/4173410929 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968effd700 1 -- 172.21.15.79:0/4173410929 >> v1:172.21.15.156:6789/0 conn(0x7f96900ca890 legacy=0x7f96900cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968effd700 1 -- 172.21.15.79:0/4173410929 >> v1:172.21.15.79:6789/0 conn(0x7f969006c050 legacy=0x7f9690071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968effd700 1 -- 172.21.15.79:0/4173410929 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969015a290 con 0x7f96900715a0 2023-06-08T03:50:35.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968dffb700 1 -- 172.21.15.79:0/4173410929 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812778445 0 0) 0x7f969006db90 con 0x7f969006c050 2023-06-08T03:50:35.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968effd700 1 --2- 172.21.15.79:0/4173410929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96900715a0 0x7f969015e3e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96840098f0 tx=0x7f9684009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5198ecaed78f53a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968dffb700 1 -- 172.21.15.79:0/4173410929 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684004230 con 0x7f96900715a0 2023-06-08T03:50:35.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968dffb700 1 -- 172.21.15.79:0/4173410929 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684004390 con 0x7f96900715a0 2023-06-08T03:50:35.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.728+0000 7f968dffb700 1 -- 172.21.15.79:0/4173410929 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96840036a0 con 0x7f96900715a0 2023-06-08T03:50:35.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 -- 172.21.15.79:0/4173410929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96900715a0 msgr2=0x7f969015e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 --2- 172.21.15.79:0/4173410929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96900715a0 0x7f969015e3e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f96840098f0 tx=0x7f9684009c00 comp rx=0 tx=0).stop 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 -- 172.21.15.79:0/4173410929 shutdown_connections 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 --2- 172.21.15.79:0/4173410929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96900715a0 0x7f969015e3e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 -- 172.21.15.79:0/4173410929 >> 172.21.15.79:0/4173410929 conn(0x7f96900ce4e0 msgr2=0x7f96900ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 -- 172.21.15.79:0/4173410929 shutdown_connections 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.729+0000 7f969574a700 1 -- 172.21.15.79:0/4173410929 wait complete. 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.730+0000 7f969574a700 1 Processor -- start 2023-06-08T03:50:35.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.730+0000 7f969574a700 1 -- start start 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 0x7f96901f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96900715a0 0x7f96900c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96900ca890 0x7f96900c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ade40 con 0x7f969006c050 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ae120 con 0x7f96900715a0 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ae400 con 0x7f96900ca890 2023-06-08T03:50:35.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 0x7f96901f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 0x7f96901f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34620/0 (socket says 172.21.15.79:34620) 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 -- 172.21.15.79:0/2894076524 learned_addr learned my addr 172.21.15.79:0/2894076524 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968f7fe700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96900ca890 0x7f96900c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968e7fc700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96900715a0 0x7f96900c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 -- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96900ca890 msgr2=0x7f96900c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96900ca890 0x7f96900c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 -- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96900715a0 msgr2=0x7f96900c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96900715a0 0x7f96900c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 -- 172.21.15.79:0/2894076524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684009550 con 0x7f969006c050 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f968effd700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 0x7f96901f1b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9684009130 tx=0x7f9684004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684003e70 con 0x7f969006c050 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968401f070 con 0x7f969006c050 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96900ae6e0 con 0x7f969006c050 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968400b8c0 con 0x7f969006c050 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.731+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96900c7df0 con 0x7f969006c050 2023-06-08T03:50:35.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.732+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9690073e90 con 0x7f969006c050 2023-06-08T03:50:35.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.733+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9684021030 con 0x7f969006c050 2023-06-08T03:50:35.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.733+0000 7f96777fe700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f967c06ace0 0x7f967c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:35.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.733+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f968408ae90 con 0x7f969006c050 2023-06-08T03:50:35.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.733+0000 7f968e7fc700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f967c06ace0 0x7f967c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:35.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.734+0000 7f968e7fc700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f967c06ace0 0x7f967c06d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96900c9a20 tx=0x7f9680009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:35.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.737+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9684055e00 con 0x7f969006c050 2023-06-08T03:50:35.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.926+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9690073e90 con 0x7f969006c050 2023-06-08T03:50:35.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.927+0000 7f96777fe700 1 -- 172.21.15.79:0/2894076524 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9684059420 con 0x7f969006c050 2023-06-08T03:50:35.931 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:50:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f967c06ace0 msgr2=0x7f967c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f967c06ace0 0x7f967c06d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96900c9a20 tx=0x7f9680009040 comp rx=0 tx=0).stop 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 msgr2=0x7f96901f1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 0x7f96901f1b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9684009130 tx=0x7f9684004750 comp rx=0 tx=0).stop 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 shutdown_connections 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f967c06ace0 0x7f967c06d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f96900ca890 0x7f96900c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f969006c050 0x7f96901f1b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 --2- 172.21.15.79:0/2894076524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f96900715a0 0x7f96900c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 >> 172.21.15.79:0/2894076524 conn(0x7f96900ce4e0 msgr2=0x7f96900cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 shutdown_connections 2023-06-08T03:50:35.935 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:35.929+0000 7f969574a700 1 -- 172.21.15.79:0/2894076524 wait complete. 2023-06-08T03:50:35.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:50:36.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.063+0000 7f1620517700 1 Processor -- start 2023-06-08T03:50:36.067 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.063+0000 7f1620517700 1 -- start start 2023-06-08T03:50:36.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f1620517700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f161806d410 0x7f161806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.068 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f1620517700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f1620517700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f161806ae60 0x7f161806b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f1620517700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161806dd20 con 0x7f161806ae60 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f1620517700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161815caa0 con 0x7f161806d410 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f1620517700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161815cd80 con 0x7f161806a860 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f161806ae60 0x7f161806b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f161806ae60 0x7f161806b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34628/0 (socket says 172.21.15.79:34628) 2023-06-08T03:50:36.069 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 -- 172.21.15.79:0/903315594 learned_addr learned my addr 172.21.15.79:0/903315594 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161e2b3700 1 --2- 172.21.15.79:0/903315594 >> v2:172.21.15.156:3300/0 conn(0x7f161806d410 0x7f161806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161dab2700 1 --2- 172.21.15.79:0/903315594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 -- 172.21.15.79:0/903315594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 msgr2=0x7f161806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 --2- 172.21.15.79:0/903315594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 -- 172.21.15.79:0/903315594 >> v2:172.21.15.156:3300/0 conn(0x7f161806d410 msgr2=0x7f161806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 --2- 172.21.15.79:0/903315594 >> v2:172.21.15.156:3300/0 conn(0x7f161806d410 0x7f161806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.070 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 -- 172.21.15.79:0/903315594 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161815d060 con 0x7f161806ae60 2023-06-08T03:50:36.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161eab4700 1 --2- 172.21.15.79:0/903315594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f161806b2e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f16180b1920 tx=0x7f161000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c08b4848efc753 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.064+0000 7f161d2b1700 1 -- 172.21.15.79:0/903315594 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000e070 con 0x7f161806ae60 2023-06-08T03:50:36.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f161d2b1700 1 -- 172.21.15.79:0/903315594 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161000b9c0 con 0x7f161806ae60 2023-06-08T03:50:36.071 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f161d2b1700 1 -- 172.21.15.79:0/903315594 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000bd70 con 0x7f161806ae60 2023-06-08T03:50:36.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 -- 172.21.15.79:0/903315594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 msgr2=0x7f161806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 --2- 172.21.15.79:0/903315594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f161806b2e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f16180b1920 tx=0x7f161000b330 comp rx=0 tx=0).stop 2023-06-08T03:50:36.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 -- 172.21.15.79:0/903315594 shutdown_connections 2023-06-08T03:50:36.073 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 --2- 172.21.15.79:0/903315594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f161806b2e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 --2- 172.21.15.79:0/903315594 >> v2:172.21.15.156:3300/0 conn(0x7f161806d410 0x7f161806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 --2- 172.21.15.79:0/903315594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 -- 172.21.15.79:0/903315594 >> 172.21.15.79:0/903315594 conn(0x7f16180ced30 msgr2=0x7f16180cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 -- 172.21.15.79:0/903315594 shutdown_connections 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.065+0000 7f1620517700 1 -- 172.21.15.79:0/903315594 wait complete. 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.066+0000 7f1620517700 1 Processor -- start 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.066+0000 7f1620517700 1 -- start start 2023-06-08T03:50:36.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f1620517700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f1620517700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f16180c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f1620517700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f161806d410 0x7f161815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f1620517700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161815b210 con 0x7f161806ae60 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f1620517700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16181f53b0 con 0x7f161806d410 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f1620517700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16181f5690 con 0x7f161806a860 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f161dab2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f16180c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f161dab2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f16180c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34636/0 (socket says 172.21.15.79:34636) 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f161dab2700 1 -- 172.21.15.79:0/341300797 learned_addr learned my addr 172.21.15.79:0/341300797 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f161e2b3700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.067+0000 7f161eab4700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f161806d410 0x7f161815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f161dab2700 1 -- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 msgr2=0x7f161815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f161dab2700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161815acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f161dab2700 1 -- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f161806d410 msgr2=0x7f161815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f161dab2700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f161806d410 0x7f161815e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f161dab2700 1 -- 172.21.15.79:0/341300797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610009b30 con 0x7f161806ae60 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f161dab2700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f16180c77b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f161400e810 tx=0x7f161400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400c070 con 0x7f161806ae60 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614013610 con 0x7f161806ae60 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16181f5970 con 0x7f161806ae60 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16140139c0 con 0x7f161806ae60 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.068+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16181f4fd0 con 0x7f161806ae60 2023-06-08T03:50:36.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.069+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16180c6010 con 0x7f161806ae60 2023-06-08T03:50:36.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.069+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1614013b20 con 0x7f161806ae60 2023-06-08T03:50:36.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.070+0000 7f160effd700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f15fc06ace0 0x7f15fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.070+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f16140894a0 con 0x7f161806ae60 2023-06-08T03:50:36.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.070+0000 7f161e2b3700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f15fc06ace0 0x7f15fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.071+0000 7f161e2b3700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f15fc06ace0 0x7f15fc06d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1608006b20 tx=0x7f1608008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.074+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1614057d00 con 0x7f161806ae60 2023-06-08T03:50:36.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.260+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f161806f1d0 con 0x7f161806ae60 2023-06-08T03:50:36.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.260+0000 7f160effd700 1 -- 172.21.15.79:0/341300797 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1614057890 con 0x7f161806ae60 2023-06-08T03:50:36.264 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:36.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f15fc06ace0 msgr2=0x7f15fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f15fc06ace0 0x7f15fc06d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1608006b20 tx=0x7f1608008040 comp rx=0 tx=0).stop 2023-06-08T03:50:36.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 msgr2=0x7f16180c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f16180c77b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f161400e810 tx=0x7f161400eb20 comp rx=0 tx=0).stop 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 shutdown_connections 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f161806d410 0x7f161815e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f15fc06ace0 0x7f15fc06d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f161806a860 0x7f161815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 --2- 172.21.15.79:0/341300797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f161806ae60 0x7f16180c77b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 >> 172.21.15.79:0/341300797 conn(0x7f16180ced30 msgr2=0x7f16180706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:36.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 shutdown_connections 2023-06-08T03:50:36.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.262+0000 7f1620517700 1 -- 172.21.15.79:0/341300797 wait complete. 2023-06-08T03:50:36.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:50:36.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 Processor -- start 2023-06-08T03:50:36.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 -- start start 2023-06-08T03:50:36.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f85d415c1f0 0x7f85d406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d40cd140 con 0x7f85d40cf130 2023-06-08T03:50:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d4071110 con 0x7f85d415c1f0 2023-06-08T03:50:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.395+0000 7f85da769700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4071be0 con 0x7f85d4071fe0 2023-06-08T03:50:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f85d415c1f0 0x7f85d406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d8d06700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f85d40cf130 0x7f85d4069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58120/0 (socket says 172.21.15.79:58120) 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f85d415c1f0 0x7f85d406d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34640/0 (socket says 172.21.15.79:34640) 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d8d06700 1 -- 172.21.15.79:0/2908228131 learned_addr learned my addr 172.21.15.79:0/2908228131 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d37fe700 1 --2- 172.21.15.79:0/2908228131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d2ffd700 1 -- 172.21.15.79:0/2908228131 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102171601 0 0) 0x7f85d40cd140 con 0x7f85d40cf130 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d2ffd700 1 -- 172.21.15.79:0/2908228131 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85bc0035b0 con 0x7f85d40cf130 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 -- 172.21.15.79:0/2908228131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 msgr2=0x7f85d40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 --2- 172.21.15.79:0/2908228131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 -- 172.21.15.79:0/2908228131 >> v1:172.21.15.79:6789/0 conn(0x7f85d40cf130 legacy=0x7f85d4069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 -- 172.21.15.79:0/2908228131 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d40cd390 con 0x7f85d415c1f0 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d3fff700 1 --2- 172.21.15.79:0/2908228131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d406d8a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009840 tx=0x7f85c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5921f10c2587d7 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d2ffd700 1 -- 172.21.15.79:0/2908228131 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4011830 con 0x7f85d415c1f0 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d2ffd700 1 -- 172.21.15.79:0/2908228131 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c4011990 con 0x7f85d415c1f0 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.396+0000 7f85d2ffd700 1 -- 172.21.15.79:0/2908228131 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c4011d20 con 0x7f85d415c1f0 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 -- 172.21.15.79:0/2908228131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 msgr2=0x7f85d406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 --2- 172.21.15.79:0/2908228131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d406d8a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009840 tx=0x7f85c4009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 -- 172.21.15.79:0/2908228131 shutdown_connections 2023-06-08T03:50:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 --2- 172.21.15.79:0/2908228131 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d406d8a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 --2- 172.21.15.79:0/2908228131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 -- 172.21.15.79:0/2908228131 >> 172.21.15.79:0/2908228131 conn(0x7f85d40b1d30 msgr2=0x7f85d406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 -- 172.21.15.79:0/2908228131 shutdown_connections 2023-06-08T03:50:36.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.397+0000 7f85da769700 1 -- 172.21.15.79:0/2908228131 wait complete. 2023-06-08T03:50:36.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.398+0000 7f85da769700 1 Processor -- start 2023-06-08T03:50:36.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.398+0000 7f85da769700 1 -- start start 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85da769700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85da769700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85d40cf130 0x7f85d40c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85da769700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d4145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85da769700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d40c32f0 con 0x7f85d415c1f0 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85da769700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4145eb0 con 0x7f85d40cf130 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85da769700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d40b4110 con 0x7f85d4071fe0 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d4145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d4145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34650/0 (socket says 172.21.15.79:34650) 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 -- 172.21.15.79:0/3450650743 learned_addr learned my addr 172.21.15.79:0/3450650743 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d3fff700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d37fe700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85d40cf130 0x7f85d40c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 -- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 msgr2=0x7f85d40b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 -- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85d40cf130 msgr2=0x7f85d40c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85d40cf130 0x7f85d40c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 -- 172.21.15.79:0/3450650743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40094a0 con 0x7f85d415c1f0 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d37fe700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85d40cf130 0x7f85d40c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.399+0000 7f85d8d06700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d4145970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00caa0 tx=0x7f85cc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.400+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc00a9e0 con 0x7f85d415c1f0 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.400+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc00ab40 con 0x7f85d415c1f0 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.400+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d40b43f0 con 0x7f85d415c1f0 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.400+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc012400 con 0x7f85d415c1f0 2023-06-08T03:50:36.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.400+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d40b49b0 con 0x7f85d415c1f0 2023-06-08T03:50:36.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.401+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d4073e90 con 0x7f85d415c1f0 2023-06-08T03:50:36.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.401+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85cc01b3b0 con 0x7f85d415c1f0 2023-06-08T03:50:36.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.402+0000 7f85d0ff9700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f85c006ace0 0x7f85c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.402+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f85cc0894c0 con 0x7f85d415c1f0 2023-06-08T03:50:36.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.402+0000 7f85d3fff700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f85c006ace0 0x7f85c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.402+0000 7f85d3fff700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f85c006ace0 0x7f85c006d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fd0 tx=0x7f85c401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.406+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85cc054310 con 0x7f85d415c1f0 2023-06-08T03:50:36.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.606+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f85d406ed70 con 0x7f85d415c1f0 2023-06-08T03:50:36.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.606+0000 7f85d0ff9700 1 -- 172.21.15.79:0/3450650743 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f85cc057930 con 0x7f85d415c1f0 2023-06-08T03:50:36.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f85c006ace0 msgr2=0x7f85c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f85c006ace0 0x7f85c006d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fd0 tx=0x7f85c401c040 comp rx=0 tx=0).stop 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 msgr2=0x7f85d4145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d4145970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00caa0 tx=0x7f85cc00cdb0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 shutdown_connections 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f85d415c1f0 0x7f85d4145970 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f85c006ace0 0x7f85c006d190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85d4071fe0 0x7f85d40b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 --2- 172.21.15.79:0/3450650743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85d40cf130 0x7f85d40c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.608+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 >> 172.21.15.79:0/3450650743 conn(0x7f85d40b1d30 msgr2=0x7f85d40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:36.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.609+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 shutdown_connections 2023-06-08T03:50:36.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.609+0000 7f85da769700 1 -- 172.21.15.79:0/3450650743 wait complete. 2023-06-08T03:50:36.614 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:50:36.624 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:50:36.624 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:50:36.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:36.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.795+0000 7f103783e700 1 Processor -- start 2023-06-08T03:50:36.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f103783e700 1 -- start start 2023-06-08T03:50:36.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f103783e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10300ca7c0 0x7f10300cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f103783e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f103006b1c0 0x7f103006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f103783e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10300cf710 con 0x7f10300707e0 2023-06-08T03:50:36.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f103783e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300cfc90 con 0x7f10300ca7c0 2023-06-08T03:50:36.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f103783e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300cf9b0 con 0x7f103006b1c0 2023-06-08T03:50:36.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.796+0000 7f10355da700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10300ca7c0 0x7f10300cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1035ddb700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f10300707e0 0x7f1030070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58134/0 (socket says 172.21.15.79:58134) 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1035ddb700 1 -- 172.21.15.79:0/2370828351 learned_addr learned my addr 172.21.15.79:0/2370828351 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1034dd9700 1 --2- 172.21.15.79:0/2370828351 >> v2:172.21.15.156:3300/0 conn(0x7f103006b1c0 0x7f103006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f10355da700 1 -- 172.21.15.79:0/2370828351 >> v2:172.21.15.156:3300/0 conn(0x7f103006b1c0 msgr2=0x7f103006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f10355da700 1 --2- 172.21.15.79:0/2370828351 >> v2:172.21.15.156:3300/0 conn(0x7f103006b1c0 0x7f103006a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f10355da700 1 -- 172.21.15.79:0/2370828351 >> v1:172.21.15.79:6789/0 conn(0x7f10300707e0 legacy=0x7f1030070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f10355da700 1 -- 172.21.15.79:0/2370828351 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030069ef0 con 0x7f10300ca7c0 2023-06-08T03:50:36.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1027fff700 1 -- 172.21.15.79:0/2370828351 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730049509 0 0) 0x7f10300cf710 con 0x7f10300707e0 2023-06-08T03:50:36.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f10355da700 1 --2- 172.21.15.79:0/2370828351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300ca7c0 0x7f10300cc900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1030070540 tx=0x7f1020009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a039027dfcc3a250 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1027fff700 1 -- 172.21.15.79:0/2370828351 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020011830 con 0x7f10300ca7c0 2023-06-08T03:50:36.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1027fff700 1 -- 172.21.15.79:0/2370828351 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020011990 con 0x7f10300ca7c0 2023-06-08T03:50:36.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.797+0000 7f1027fff700 1 -- 172.21.15.79:0/2370828351 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020011d20 con 0x7f10300ca7c0 2023-06-08T03:50:36.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 -- 172.21.15.79:0/2370828351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300ca7c0 msgr2=0x7f10300cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 --2- 172.21.15.79:0/2370828351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300ca7c0 0x7f10300cc900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1030070540 tx=0x7f1020009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 -- 172.21.15.79:0/2370828351 shutdown_connections 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 --2- 172.21.15.79:0/2370828351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300ca7c0 0x7f10300cc900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 --2- 172.21.15.79:0/2370828351 >> v2:172.21.15.156:3300/0 conn(0x7f103006b1c0 0x7f103006a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 -- 172.21.15.79:0/2370828351 >> 172.21.15.79:0/2370828351 conn(0x7f10300b14a0 msgr2=0x7f10300cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 -- 172.21.15.79:0/2370828351 shutdown_connections 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.798+0000 7f103783e700 1 -- 172.21.15.79:0/2370828351 wait complete. 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.799+0000 7f103783e700 1 Processor -- start 2023-06-08T03:50:36.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.799+0000 7f103783e700 1 -- start start 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f103006b1c0 0x7f10300b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 0x7f1030153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10300ca7c0 0x7f10301f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1030153dd0 con 0x7f10300707e0 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300b5070 con 0x7f103006b1c0 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10300b5350 con 0x7f10300ca7c0 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 0x7f1030153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 0x7f1030153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34662/0 (socket says 172.21.15.79:34662) 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 -- 172.21.15.79:0/2629794255 learned_addr learned my addr 172.21.15.79:0/2629794255 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f10355da700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f103006b1c0 0x7f10300b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1035ddb700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10300ca7c0 0x7f10301f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 -- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10300ca7c0 msgr2=0x7f10301f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10300ca7c0 0x7f10301f5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 -- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f103006b1c0 msgr2=0x7f10300b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:36.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f103006b1c0 0x7f10300b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 -- 172.21.15.79:0/2629794255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10200094a0 con 0x7f10300707e0 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f10355da700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f103006b1c0 0x7f10300b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1035ddb700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10300ca7c0 0x7f10301f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1034dd9700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 0x7f1030153890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e810 tx=0x7f102c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c013610 con 0x7f10300707e0 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c013770 con 0x7f10300707e0 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10300b5630 con 0x7f10300707e0 2023-06-08T03:50:36.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.800+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c013b20 con 0x7f10300707e0 2023-06-08T03:50:36.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.801+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10300c7e80 con 0x7f10300707e0 2023-06-08T03:50:36.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.801+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10300b4c30 con 0x7f10300707e0 2023-06-08T03:50:36.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.802+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f102c013c80 con 0x7f10300707e0 2023-06-08T03:50:36.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.802+0000 7f1025ffb700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:36.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.802+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f102c089420 con 0x7f10300707e0 2023-06-08T03:50:36.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.803+0000 7f10355da700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:36.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.803+0000 7f10355da700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f101406ace0 0x7f101406d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10200090b0 tx=0x7f102001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:36.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:36.807+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f102c057c80 con 0x7f10300707e0 2023-06-08T03:50:37.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.002+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10300726e0 con 0x7f10300707e0 2023-06-08T03:50:37.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.003+0000 7f1025ffb700 1 -- 172.21.15.79:0/2629794255 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f102c057810 con 0x7f10300707e0 2023-06-08T03:50:37.007 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:37.007 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:37.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f101406ace0 msgr2=0x7f101406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f101406ace0 0x7f101406d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10200090b0 tx=0x7f102001c040 comp rx=0 tx=0).stop 2023-06-08T03:50:37.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 msgr2=0x7f1030153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 0x7f1030153890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e810 tx=0x7f102c00ebd0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 shutdown_connections 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10300ca7c0 0x7f10301f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f103006b1c0 0x7f10300b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 --2- 172.21.15.79:0/2629794255 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10300707e0 0x7f1030153890 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 >> 172.21.15.79:0/2629794255 conn(0x7f10300b14a0 msgr2=0x7f103006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 shutdown_connections 2023-06-08T03:50:37.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.006+0000 7f103783e700 1 -- 172.21.15.79:0/2629794255 wait complete. 2023-06-08T03:50:37.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:50:37.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:50:37.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.139+0000 7fd097078700 1 Processor -- start 2023-06-08T03:50:37.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.139+0000 7fd097078700 1 -- start start 2023-06-08T03:50:37.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.140+0000 7fd097078700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd09015c1f0 0x7fd09006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.140+0000 7fd097078700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0900cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.140+0000 7fd097078700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900cd140 con 0x7fd0900cf130 2023-06-08T03:50:37.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.140+0000 7fd097078700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd090071110 con 0x7fd09015c1f0 2023-06-08T03:50:37.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.140+0000 7fd097078700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd090071be0 con 0x7fd090071fe0 2023-06-08T03:50:37.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0900cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd094e14700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd09015c1f0 0x7fd09006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd095615700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd0900cf130 0x7fd090069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60308/0 (socket says 172.21.15.79:60308) 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0900cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:48992/0 (socket says 172.21.15.79:48992) 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd095615700 1 -- 172.21.15.79:0/2348610964 learned_addr learned my addr 172.21.15.79:0/2348610964 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 -- 172.21.15.79:0/2348610964 >> v2:172.21.15.156:3300/0 conn(0x7fd09015c1f0 msgr2=0x7fd09006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 --2- 172.21.15.79:0/2348610964 >> v2:172.21.15.156:3300/0 conn(0x7fd09015c1f0 0x7fd09006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 -- 172.21.15.79:0/2348610964 >> v1:172.21.15.156:6789/0 conn(0x7fd0900cf130 legacy=0x7fd090069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 -- 172.21.15.79:0/2348610964 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0900cd390 con 0x7fd090071fe0 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd094e14700 1 --2- 172.21.15.79:0/2348610964 >> v2:172.21.15.156:3300/0 conn(0x7fd09015c1f0 0x7fd09006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08ffff700 1 --2- 172.21.15.79:0/2348610964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0900cebf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009840 tx=0x7fd07c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ace575df65c10f9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.141+0000 7fd08f7fe700 1 -- 172.21.15.79:0/2348610964 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011830 con 0x7fd090071fe0 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd08f7fe700 1 -- 172.21.15.79:0/2348610964 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c011990 con 0x7fd090071fe0 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd08f7fe700 1 -- 172.21.15.79:0/2348610964 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011d20 con 0x7fd090071fe0 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 -- 172.21.15.79:0/2348610964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 msgr2=0x7fd0900cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 --2- 172.21.15.79:0/2348610964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0900cebf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009840 tx=0x7fd07c009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:37.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 -- 172.21.15.79:0/2348610964 shutdown_connections 2023-06-08T03:50:37.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 --2- 172.21.15.79:0/2348610964 >> v2:172.21.15.156:3300/0 conn(0x7fd09015c1f0 0x7fd09006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 --2- 172.21.15.79:0/2348610964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0900cebf0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 -- 172.21.15.79:0/2348610964 >> 172.21.15.79:0/2348610964 conn(0x7fd0900b1d30 msgr2=0x7fd09006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:37.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 -- 172.21.15.79:0/2348610964 shutdown_connections 2023-06-08T03:50:37.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.142+0000 7fd097078700 1 -- 172.21.15.79:0/2348610964 wait complete. 2023-06-08T03:50:37.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.144+0000 7fd097078700 1 Processor -- start 2023-06-08T03:50:37.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.144+0000 7fd097078700 1 -- start start 2023-06-08T03:50:37.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0901ef100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0900cf130 0x7fd0900b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 0x7fd09015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090152cc0 con 0x7fd09015c1f0 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd090152fa0 con 0x7fd0900cf130 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd090153280 con 0x7fd090071fe0 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 0x7fd09015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 0x7fd09015ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34680/0 (socket says 172.21.15.79:34680) 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 -- 172.21.15.79:0/1830405638 learned_addr learned my addr 172.21.15.79:0/1830405638 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd094e14700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0901ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd08ffff700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0900cf130 0x7fd0900b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 -- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 msgr2=0x7fd0901ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0901ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 -- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0900cf130 msgr2=0x7fd0900b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0900cf130 0x7fd0900b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 -- 172.21.15.79:0/1830405638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0840094a0 con 0x7fd09015c1f0 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd08ffff700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0900cf130 0x7fd0900b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd095615700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 0x7fd09015ceb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ca80 tx=0x7fd08800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08800a9e0 con 0x7fd09015c1f0 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08800ab40 con 0x7fd09015c1f0 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c0094a0 con 0x7fd09015c1f0 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.145+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088012400 con 0x7fd09015c1f0 2023-06-08T03:50:37.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.146+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd090153840 con 0x7fd09015c1f0 2023-06-08T03:50:37.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.147+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0900666a0 con 0x7fd09015c1f0 2023-06-08T03:50:37.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.147+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd08801b3b0 con 0x7fd09015c1f0 2023-06-08T03:50:37.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.147+0000 7fd08d7fa700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd07406ace0 0x7fd07406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.147+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd0880891e0 con 0x7fd09015c1f0 2023-06-08T03:50:37.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.148+0000 7fd094e14700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd07406ace0 0x7fd07406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.148+0000 7fd094e14700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd07406ace0 0x7fd07406d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd084009cd0 tx=0x7fd084009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.152+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0880540e0 con 0x7fd09015c1f0 2023-06-08T03:50:37.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.338+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd090153a30 con 0x7fd09015c1f0 2023-06-08T03:50:37.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.338+0000 7fd08d7fa700 1 -- 172.21.15.79:0/1830405638 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd088057700 con 0x7fd09015c1f0 2023-06-08T03:50:37.342 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd07406ace0 msgr2=0x7fd07406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd07406ace0 0x7fd07406d190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd084009cd0 tx=0x7fd084009210 comp rx=0 tx=0).stop 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 msgr2=0x7fd09015ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 0x7fd09015ceb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ca80 tx=0x7fd08800cd90 comp rx=0 tx=0).stop 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 shutdown_connections 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd09015c1f0 0x7fd09015ceb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd07406ace0 0x7fd07406d190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd090071fe0 0x7fd0901ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 --2- 172.21.15.79:0/1830405638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0900cf130 0x7fd0900b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 >> 172.21.15.79:0/1830405638 conn(0x7fd0900b1d30 msgr2=0x7fd09015ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:37.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 shutdown_connections 2023-06-08T03:50:37.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.340+0000 7fd097078700 1 -- 172.21.15.79:0/1830405638 wait complete. 2023-06-08T03:50:37.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:50:37.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.474+0000 7fec5f22e700 1 Processor -- start 2023-06-08T03:50:37.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.474+0000 7fec5f22e700 1 -- start start 2023-06-08T03:50:37.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.474+0000 7fec5f22e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 0x7fec5806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5f22e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fec58069d50 0x7fec5806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5f22e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec580cd140 con 0x7fec58071fe0 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5f22e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec5806a710 con 0x7fec58069d50 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5f22e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec58071110 con 0x7fec5815c1f0 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec57fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fec58071fe0 0x7fec580cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58162/0 (socket says 172.21.15.79:58162) 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec57fff700 1 -- 172.21.15.79:0/1766935653 learned_addr learned my addr 172.21.15.79:0/1766935653 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:37.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5cfca700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 0x7fec5806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502635255 0 0) 0x7fec580cd140 con 0x7fec58071fe0 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec480035b0 con 0x7fec58071fe0 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5d7cb700 1 --2- 172.21.15.79:0/1766935653 >> v2:172.21.15.156:3300/0 conn(0x7fec58069d50 0x7fec5806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2613760508 0 0) 0x7fec480035b0 con 0x7fec58071fe0 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 msgr2=0x7fec5806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 0x7fec5806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 >> v2:172.21.15.156:3300/0 conn(0x7fec58069d50 msgr2=0x7fec5806a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 --2- 172.21.15.79:0/1766935653 >> v2:172.21.15.156:3300/0 conn(0x7fec58069d50 0x7fec5806a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec58071be0 con 0x7fec58071fe0 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fec44001670 con 0x7fec58071fe0 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 >> v1:172.21.15.79:6789/0 conn(0x7fec58071fe0 legacy=0x7fec580cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec48004850 0x7fec48004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 0x7fec48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5806a710 con 0x7fec48003a00 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58071110 con 0x7fec48004850 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4800ba90 con 0x7fec5815c1f0 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec5cfca700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.475+0000 7fec57fff700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec48004850 0x7fec48004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5d7cb700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 0x7fec48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5cfca700 1 -- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 msgr2=0x7fec48005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5cfca700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815c1f0 0x7fec48005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5cfca700 1 -- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec48004850 msgr2=0x7fec48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5cfca700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec48004850 0x7fec48004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5cfca700 1 -- 172.21.15.79:0/1766935653 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec480035b0 con 0x7fec48003a00 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5cfca700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec48003e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009b60 tx=0x7fec48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fd31433768dfe4 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00e6c0 con 0x7fec48003a00 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c00e820 con 0x7fec48003a00 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec577fe700 1 -- 172.21.15.79:0/1766935653 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00eb90 con 0x7fec48003a00 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5f22e700 1 -- 172.21.15.79:0/1766935653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 msgr2=0x7fec48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5f22e700 1 --2- 172.21.15.79:0/1766935653 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec48003e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009b60 tx=0x7fec48005880 comp rx=0 tx=0).stop 2023-06-08T03:50:37.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5d7cb700 1 -- 172.21.15.79:0/1766935653 reap_dead start 2023-06-08T03:50:37.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5f22e700 1 -- 172.21.15.79:0/1766935653 shutdown_connections 2023-06-08T03:50:37.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.476+0000 7fec5f22e700 1 -- 172.21.15.79:0/1766935653 >> 172.21.15.79:0/1766935653 conn(0x7fec580b1d30 msgr2=0x7fec5806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:37.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.477+0000 7fec5f22e700 1 -- 172.21.15.79:0/1766935653 shutdown_connections 2023-06-08T03:50:37.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.477+0000 7fec5f22e700 1 -- 172.21.15.79:0/1766935653 wait complete. 2023-06-08T03:50:37.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.478+0000 7fec5f22e700 1 Processor -- start 2023-06-08T03:50:37.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.478+0000 7fec5f22e700 1 -- start start 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5f22e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec580c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5f22e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5815abc0 0x7fec58151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5f22e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815b030 0x7fec58151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5f22e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581520f0 con 0x7fec48003a00 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5f22e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec581f6060 con 0x7fec5815abc0 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5f22e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec581f54e0 con 0x7fec5815b030 2023-06-08T03:50:37.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5cfca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec580c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5cfca700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec580c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34696/0 (socket says 172.21.15.79:34696) 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5cfca700 1 -- 172.21.15.79:0/249409757 learned_addr learned my addr 172.21.15.79:0/249409757 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec5d7cb700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815b030 0x7fec58151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.479+0000 7fec57fff700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5815abc0 0x7fec58151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5cfca700 1 -- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815b030 msgr2=0x7fec58151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5cfca700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815b030 0x7fec58151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.490 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5cfca700 1 -- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5815abc0 msgr2=0x7fec58151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5cfca700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5815abc0 0x7fec58151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5cfca700 1 -- 172.21.15.79:0/249409757 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec581f57c0 con 0x7fec48003a00 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec57fff700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5815abc0 0x7fec58151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5d7cb700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815b030 0x7fec58151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5cfca700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec580c36e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec4c000c00 tx=0x7fec4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00e6c0 con 0x7fec48003a00 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c003ad0 con 0x7fec48003a00 2023-06-08T03:50:37.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec581f5aa0 con 0x7fec48003a00 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0045d0 con 0x7fec48003a00 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.480+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5815d340 con 0x7fec48003a00 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.481+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec58073e90 con 0x7fec48003a00 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.481+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec4c013470 con 0x7fec48003a00 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.482+0000 7fec557fa700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fec3c06ace0 0x7fec3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.482+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fec4c08b260 con 0x7fec48003a00 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.482+0000 7fec57fff700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fec3c06ace0 0x7fec3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.482+0000 7fec57fff700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fec3c06ace0 0x7fec3c06d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec44001120 tx=0x7fec44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.486+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec4c056010 con 0x7fec48003a00 2023-06-08T03:50:37.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.669+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec5806ed70 con 0x7fec48003a00 2023-06-08T03:50:37.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.669+0000 7fec557fa700 1 -- 172.21.15.79:0/249409757 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec4c059630 con 0x7fec48003a00 2023-06-08T03:50:37.674 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:50:37.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fec3c06ace0 msgr2=0x7fec3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fec3c06ace0 0x7fec3c06d190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec44001120 tx=0x7fec44009040 comp rx=0 tx=0).stop 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 msgr2=0x7fec580c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec580c36e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fec4c000c00 tx=0x7fec4c0047c0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 shutdown_connections 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fec3c06ace0 0x7fec3c06d190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fec5815b030 0x7fec58151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec48003a00 0x7fec580c36e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 --2- 172.21.15.79:0/249409757 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec5815abc0 0x7fec58151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 >> 172.21.15.79:0/249409757 conn(0x7fec580b1d30 msgr2=0x7fec580b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:37.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 shutdown_connections 2023-06-08T03:50:37.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.671+0000 7fec5f22e700 1 -- 172.21.15.79:0/249409757 wait complete. 2023-06-08T03:50:37.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:50:37.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f157a5dc700 1 Processor -- start 2023-06-08T03:50:37.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f157a5dc700 1 -- start start 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f157a5dc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f157406a460 0x7f1574071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f157a5dc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15740cd140 con 0x7f157406d8a0 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f157a5dc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1574071110 con 0x7f157415c1f0 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f157a5dc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1574071be0 con 0x7f157406a460 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.806+0000 7f15737fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f157406d8a0 0x7f1574069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58182/0 (socket says 172.21.15.79:58182) 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f15737fe700 1 -- 172.21.15.79:0/3935414473 learned_addr learned my addr 172.21.15.79:0/3935414473 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:37.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1578b79700 1 --2- 172.21.15.79:0/3935414473 >> v2:172.21.15.156:3300/0 conn(0x7f157406a460 0x7f1574071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233269400 0 0) 0x7f15740cd140 con 0x7f157406d8a0 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15600035b0 con 0x7f157406d8a0 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986386402 0 0) 0x7f1574071110 con 0x7f157415c1f0 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15740cd140 con 0x7f157415c1f0 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3793749998 0 0) 0x7f15600035b0 con 0x7f157406d8a0 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 >> v2:172.21.15.156:3300/0 conn(0x7f157406a460 msgr2=0x7f1574071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 --2- 172.21.15.79:0/3935414473 >> v2:172.21.15.156:3300/0 conn(0x7f157406a460 0x7f1574071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 >> v1:172.21.15.156:6789/0 conn(0x7f157415c1f0 legacy=0x7f15740cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740cd390 con 0x7f157406d8a0 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f155c0015f0 con 0x7f157406d8a0 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1578b79700 1 --2- 172.21.15.79:0/3935414473 >> v2:172.21.15.156:3300/0 conn(0x7f157406a460 0x7f1574071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 >> v1:172.21.15.79:6789/0 conn(0x7f157406d8a0 legacy=0x7f1574069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560003be0 0x7f1560003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1560004850 0x7f1560004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15600052c0 0x7f156000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1574071be0 con 0x7f15600052c0 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156000c200 con 0x7f1560003be0 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156000c3c0 con 0x7f1560004850 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1578b79700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15600052c0 0x7f156000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1573fff700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560003be0 0x7f1560003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f15737fe700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1560004850 0x7f1560004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1578b79700 1 -- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1560004850 msgr2=0x7f1560004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.807+0000 7f1578b79700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1560004850 0x7f1560004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1578b79700 1 -- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560003be0 msgr2=0x7f1560003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1578b79700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560003be0 0x7f1560003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1578b79700 1 -- 172.21.15.79:0/3935414473 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15600035b0 con 0x7f15600052c0 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1573fff700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1560003be0 0x7f1560003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1578b79700 1 -- 172.21.15.79:0/3935414473 reap_dead start 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1578b79700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15600052c0 0x7f156000ba10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e320 tx=0x7f1560005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf6e76f37cf6148 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00edc0 con 0x7f15600052c0 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00f040 con 0x7f15600052c0 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f1572ffd700 1 -- 172.21.15.79:0/3935414473 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0146c0 con 0x7f15600052c0 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 -- 172.21.15.79:0/3935414473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15600052c0 msgr2=0x7f156000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15600052c0 0x7f156000ba10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e320 tx=0x7f1560005880 comp rx=0 tx=0).stop 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 -- 172.21.15.79:0/3935414473 shutdown_connections 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3935414473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15600052c0 0x7f156000ba10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 -- 172.21.15.79:0/3935414473 >> 172.21.15.79:0/3935414473 conn(0x7f15740b1d30 msgr2=0x7f157406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:37.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 -- 172.21.15.79:0/3935414473 shutdown_connections 2023-06-08T03:50:37.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.808+0000 7f157a5dc700 1 -- 172.21.15.79:0/3935414473 wait complete. 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.810+0000 7f157a5dc700 1 Processor -- start 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.810+0000 7f157a5dc700 1 -- start start 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.810+0000 7f157a5dc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15600052c0 0x7f15740c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157415a910 0x7f157414d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 0x7f15740cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157415b180 con 0x7f15740b2c40 2023-06-08T03:50:37.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15740c8e20 con 0x7f157415a910 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15740c9100 con 0x7f15600052c0 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 0x7f15740cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 0x7f15740cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34720/0 (socket says 172.21.15.79:34720) 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 -- 172.21.15.79:0/3238581561 learned_addr learned my addr 172.21.15.79:0/3238581561 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1573fff700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15600052c0 0x7f15740c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f15737fe700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157415a910 0x7f157414d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 -- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15600052c0 msgr2=0x7f15740c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15600052c0 0x7f15740c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 -- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157415a910 msgr2=0x7f157414d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157415a910 0x7f157414d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 -- 172.21.15.79:0/3238581561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740c93e0 con 0x7f15740b2c40 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1578b79700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 0x7f15740cee90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1574070e40 tx=0x7f156c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004450 con 0x7f15740b2c40 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c0045d0 con 0x7f15740b2c40 2023-06-08T03:50:37.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15740b2440 con 0x7f15740b2c40 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c014480 con 0x7f15740b2c40 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.811+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157414de00 con 0x7f15740b2c40 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.812+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15740b2960 con 0x7f15740b2c40 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.813+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f156c023070 con 0x7f15740b2c40 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.813+0000 7f1570ff9700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f155406ace0 0x7f155406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.813+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f156c08b070 con 0x7f15740b2c40 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.813+0000 7f1573fff700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f155406ace0 0x7f155406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:37.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.814+0000 7f1573fff700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f155406ace0 0x7f155406d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15640010e0 tx=0x7f15640012d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:37.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:37.817+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f15740b2960 con 0x7f15740b2c40 2023-06-08T03:50:38.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.005+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15740b2960 con 0x7f15740b2c40 2023-06-08T03:50:38.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.005+0000 7f1570ff9700 1 -- 172.21.15.79:0/3238581561 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15740b2960 con 0x7f15740b2c40 2023-06-08T03:50:38.009 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:38.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f155406ace0 msgr2=0x7f155406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f155406ace0 0x7f155406d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15640010e0 tx=0x7f15640012d0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 msgr2=0x7f15740cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 0x7f15740cee90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1574070e40 tx=0x7f156c009750 comp rx=0 tx=0).stop 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 shutdown_connections 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f15740b2c40 0x7f15740cee90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f155406ace0 0x7f155406d190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15600052c0 0x7f15740c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 --2- 172.21.15.79:0/3238581561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f157415a910 0x7f157414d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 >> 172.21.15.79:0/3238581561 conn(0x7f15740b1d30 msgr2=0x7f15740726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:38.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 shutdown_connections 2023-06-08T03:50:38.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.007+0000 7f157a5dc700 1 -- 172.21.15.79:0/3238581561 wait complete. 2023-06-08T03:50:38.023 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:50:38.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.140+0000 7f4fa7950700 1 Processor -- start 2023-06-08T03:50:38.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.141+0000 7f4fa7950700 1 -- start start 2023-06-08T03:50:38.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.141+0000 7f4fa7950700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 0x7f4fa00af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.141+0000 7f4fa7950700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.141+0000 7f4fa7950700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa00cae50 con 0x7f4fa00b20f0 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.141+0000 7f4fa7950700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa00cbea0 con 0x7f4fa00d0060 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.141+0000 7f4fa7950700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00cda30 con 0x7f4fa00cd080 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa56ec700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 0x7f4fa00af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa5eed700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4fa00b20f0 0x7f4fa00b2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60364/0 (socket says 172.21.15.79:60364) 2023-06-08T03:50:38.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa00cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49046/0 (socket says 172.21.15.79:49046) 2023-06-08T03:50:38.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa5eed700 1 -- 172.21.15.79:0/1362972661 learned_addr learned my addr 172.21.15.79:0/1362972661 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:38.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa56ec700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 0x7f4fa00af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:57618/0 (socket says 172.21.15.79:57618) 2023-06-08T03:50:38.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 -- 172.21.15.79:0/1362972661 >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 msgr2=0x7f4fa00af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:38.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 --2- 172.21.15.79:0/1362972661 >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 0x7f4fa00af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 -- 172.21.15.79:0/1362972661 >> v1:172.21.15.156:6789/0 conn(0x7f4fa00b20f0 legacy=0x7f4fa00b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4f97fff700 1 -- 172.21.15.79:0/1362972661 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820313371 0 0) 0x7f4fa00cae50 con 0x7f4fa00b20f0 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 -- 172.21.15.79:0/1362972661 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa015d930 con 0x7f4fa00cd080 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa56ec700 1 --2- 172.21.15.79:0/1362972661 >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 0x7f4fa00af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.142+0000 7f4fa4eeb700 1 --2- 172.21.15.79:0/1362972661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa00cd4f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009840 tx=0x7f4f9c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=570108892dac3417 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4f97fff700 1 -- 172.21.15.79:0/1362972661 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c011830 con 0x7f4fa00cd080 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4f97fff700 1 -- 172.21.15.79:0/1362972661 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c011990 con 0x7f4fa00cd080 2023-06-08T03:50:38.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4f97fff700 1 -- 172.21.15.79:0/1362972661 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c011d20 con 0x7f4fa00cd080 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 -- 172.21.15.79:0/1362972661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 msgr2=0x7f4fa00cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 --2- 172.21.15.79:0/1362972661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa00cd4f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009840 tx=0x7f4f9c009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 -- 172.21.15.79:0/1362972661 shutdown_connections 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 --2- 172.21.15.79:0/1362972661 >> v2:172.21.15.156:3300/0 conn(0x7f4fa00d0060 0x7f4fa00af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 --2- 172.21.15.79:0/1362972661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa00cd4f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 -- 172.21.15.79:0/1362972661 >> 172.21.15.79:0/1362972661 conn(0x7f4fa006e1e0 msgr2=0x7f4fa006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:38.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 -- 172.21.15.79:0/1362972661 shutdown_connections 2023-06-08T03:50:38.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.143+0000 7f4fa7950700 1 -- 172.21.15.79:0/1362972661 wait complete. 2023-06-08T03:50:38.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.144+0000 7f4fa7950700 1 Processor -- start 2023-06-08T03:50:38.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.144+0000 7f4fa7950700 1 -- start start 2023-06-08T03:50:38.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa7950700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 0x7f4fa0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa7950700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa0152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa7950700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa00d0060 0x7f4fa006ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa7950700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa01527d0 con 0x7f4fa00b20f0 2023-06-08T03:50:38.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa7950700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00ca030 con 0x7f4fa00d0060 2023-06-08T03:50:38.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa7950700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00c66a0 con 0x7f4fa00cd080 2023-06-08T03:50:38.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa56ec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 0x7f4fa0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa56ec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 0x7f4fa0152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34728/0 (socket says 172.21.15.79:34728) 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa56ec700 1 -- 172.21.15.79:0/3166202440 learned_addr learned my addr 172.21.15.79:0/3166202440 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa4eeb700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa0152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.145+0000 7f4fa5eed700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa00d0060 0x7f4fa006ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa56ec700 1 -- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 msgr2=0x7f4fa0152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa56ec700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa0152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa56ec700 1 -- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa00d0060 msgr2=0x7f4fa006ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa56ec700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa00d0060 0x7f4fa006ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa56ec700 1 -- 172.21.15.79:0/3166202440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900094a0 con 0x7f4fa00b20f0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa56ec700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 0x7f4fa0152940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000ebe0 tx=0x7f4f9000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9000c850 con 0x7f4fa00b20f0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9000c9b0 con 0x7f4fa00b20f0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c0094a0 con 0x7f4fa00b20f0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9000cd40 con 0x7f4fa00b20f0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.146+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa01e3a40 con 0x7f4fa00b20f0 2023-06-08T03:50:38.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.147+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa0073e90 con 0x7f4fa00b20f0 2023-06-08T03:50:38.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.147+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4f90013470 con 0x7f4fa00b20f0 2023-06-08T03:50:38.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.148+0000 7f4f95ffb700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8406ace0 0x7f4f8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.148+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4f900897a0 con 0x7f4fa00b20f0 2023-06-08T03:50:38.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.148+0000 7f4fa4eeb700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8406ace0 0x7f4f8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.148+0000 7f4fa4eeb700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8406ace0 0x7f4f8406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009840 tx=0x7f4f9c01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.152+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4f90054560 con 0x7f4fa00b20f0 2023-06-08T03:50:38.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.337+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fa0070120 con 0x7f4fa00b20f0 2023-06-08T03:50:38.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.337+0000 7f4f95ffb700 1 -- 172.21.15.79:0/3166202440 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f9000cea0 con 0x7f4fa00b20f0 2023-06-08T03:50:38.342 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:50:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8406ace0 msgr2=0x7f4f8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8406ace0 0x7f4f8406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009840 tx=0x7f4f9c01c040 comp rx=0 tx=0).stop 2023-06-08T03:50:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 msgr2=0x7f4fa0152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 0x7f4fa0152940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000ebe0 tx=0x7f4f9000eef0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 shutdown_connections 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fa00d0060 0x7f4fa006ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4fa00b20f0 0x7f4fa0152940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8406ace0 0x7f4f8406d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 --2- 172.21.15.79:0/3166202440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fa00cd080 0x7f4fa0152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 >> 172.21.15.79:0/3166202440 conn(0x7f4fa006e1e0 msgr2=0x7f4fa00cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 shutdown_connections 2023-06-08T03:50:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.339+0000 7f4fa7950700 1 -- 172.21.15.79:0/3166202440 wait complete. 2023-06-08T03:50:38.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:50:38.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.472+0000 7f1d959a5700 1 Processor -- start 2023-06-08T03:50:38.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d959a5700 1 -- start start 2023-06-08T03:50:38.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d959a5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1d9015c280 0x7f1d900d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d959a5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d900ceb40 0x7f1d900d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d959a5700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900cdaa0 con 0x7f1d900b1990 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d959a5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900cdd80 con 0x7f1d9015c280 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d959a5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900ce060 con 0x7f1d900ceb40 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d8effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1d9015c280 0x7f1d900d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d8effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1d9015c280 0x7f1d900d23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34736/0 (socket says 172.21.15.79:34736) 2023-06-08T03:50:38.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.473+0000 7f1d8effd700 1 -- 172.21.15.79:0/2200567380 learned_addr learned my addr 172.21.15.79:0/2200567380 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:38.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8e7fc700 1 --2- 172.21.15.79:0/2200567380 >> v2:172.21.15.156:3300/0 conn(0x7f1d900ceb40 0x7f1d900d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8effd700 1 -- 172.21.15.79:0/2200567380 >> v2:172.21.15.156:3300/0 conn(0x7f1d900ceb40 msgr2=0x7f1d900d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:38.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8effd700 1 --2- 172.21.15.79:0/2200567380 >> v2:172.21.15.156:3300/0 conn(0x7f1d900ceb40 0x7f1d900d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8effd700 1 -- 172.21.15.79:0/2200567380 >> v1:172.21.15.156:6789/0 conn(0x7f1d900b1990 legacy=0x7f1d9015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8effd700 1 -- 172.21.15.79:0/2200567380 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900ce340 con 0x7f1d9015c280 2023-06-08T03:50:38.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8e7fc700 1 --2- 172.21.15.79:0/2200567380 >> v2:172.21.15.156:3300/0 conn(0x7f1d900ceb40 0x7f1d900d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8effd700 1 --2- 172.21.15.79:0/2200567380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9015c280 0x7f1d900d23f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d900664e0 tx=0x7f1d84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d51de257e581920 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8dffb700 1 -- 172.21.15.79:0/2200567380 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84005e30 con 0x7f1d9015c280 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8dffb700 1 -- 172.21.15.79:0/2200567380 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84004030 con 0x7f1d9015c280 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d8dffb700 1 -- 172.21.15.79:0/2200567380 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840043e0 con 0x7f1d9015c280 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d959a5700 1 -- 172.21.15.79:0/2200567380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9015c280 msgr2=0x7f1d900d23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.474+0000 7f1d959a5700 1 --2- 172.21.15.79:0/2200567380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9015c280 0x7f1d900d23f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d900664e0 tx=0x7f1d84009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.475+0000 7f1d959a5700 1 -- 172.21.15.79:0/2200567380 shutdown_connections 2023-06-08T03:50:38.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.475+0000 7f1d959a5700 1 --2- 172.21.15.79:0/2200567380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d9015c280 0x7f1d900d23f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.475+0000 7f1d959a5700 1 --2- 172.21.15.79:0/2200567380 >> v2:172.21.15.156:3300/0 conn(0x7f1d900ceb40 0x7f1d900d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.475+0000 7f1d959a5700 1 -- 172.21.15.79:0/2200567380 >> 172.21.15.79:0/2200567380 conn(0x7f1d9009bc40 msgr2=0x7f1d900d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:38.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.475+0000 7f1d959a5700 1 -- 172.21.15.79:0/2200567380 shutdown_connections 2023-06-08T03:50:38.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.475+0000 7f1d959a5700 1 -- 172.21.15.79:0/2200567380 wait complete. 2023-06-08T03:50:38.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.476+0000 7f1d959a5700 1 Processor -- start 2023-06-08T03:50:38.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.476+0000 7f1d959a5700 1 -- start start 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 0x7f1d900b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d900ceb40 0x7f1d900b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d9015c280 0x7f1d900b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900b2a60 con 0x7f1d900b1990 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900b2d40 con 0x7f1d9015c280 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900b3020 con 0x7f1d900ceb40 2023-06-08T03:50:38.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 0x7f1d900b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 0x7f1d900b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34740/0 (socket says 172.21.15.79:34740) 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 -- 172.21.15.79:0/3576783219 learned_addr learned my addr 172.21.15.79:0/3576783219 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8e7fc700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d900ceb40 0x7f1d900b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8f7fe700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d9015c280 0x7f1d900b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 -- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d900ceb40 msgr2=0x7f1d900b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d900ceb40 0x7f1d900b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 -- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d9015c280 msgr2=0x7f1d900b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d9015c280 0x7f1d900b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 -- 172.21.15.79:0/3576783219 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009500 con 0x7f1d900b1990 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8e7fc700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d900ceb40 0x7f1d900b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8f7fe700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d9015c280 0x7f1d900b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d8effd700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 0x7f1d900b20f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009130 tx=0x7f1d84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84003e10 con 0x7f1d900b1990 2023-06-08T03:50:38.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8401d430 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8401d7e0 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d900b3300 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.477+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d900b2630 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.478+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d900b27c0 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.479+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d8401d940 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.479+0000 7f1d77fff700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d7c06ace0 0x7f1d7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.479+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1d8408ace0 con 0x7f1d900b1990 2023-06-08T03:50:38.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.480+0000 7f1d8e7fc700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d7c06ace0 0x7f1d7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.480+0000 7f1d8e7fc700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d7c06ace0 0x7f1d7c06d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d9015fce0 tx=0x7f1d80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.484+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d84055b30 con 0x7f1d900b1990 2023-06-08T03:50:38.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.668+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1d90073e90 con 0x7f1d900b1990 2023-06-08T03:50:38.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.669+0000 7f1d77fff700 1 -- 172.21.15.79:0/3576783219 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1d84059150 con 0x7f1d900b1990 2023-06-08T03:50:38.673 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:38.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.670+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d7c06ace0 msgr2=0x7f1d7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.670+0000 7f1d959a5700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d7c06ace0 0x7f1d7c06d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1d9015fce0 tx=0x7f1d80008040 comp rx=0 tx=0).stop 2023-06-08T03:50:38.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 msgr2=0x7f1d900b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 0x7f1d900b20f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009130 tx=0x7f1d84004750 comp rx=0 tx=0).stop 2023-06-08T03:50:38.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 shutdown_connections 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d9015c280 0x7f1d900b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1d900b1990 0x7f1d900b20f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d7c06ace0 0x7f1d7c06d190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 --2- 172.21.15.79:0/3576783219 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d900ceb40 0x7f1d900b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 >> 172.21.15.79:0/3576783219 conn(0x7f1d9009bc40 msgr2=0x7f1d900af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 shutdown_connections 2023-06-08T03:50:38.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.671+0000 7f1d959a5700 1 -- 172.21.15.79:0/3576783219 wait complete. 2023-06-08T03:50:38.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:50:38.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.802+0000 7f056a020700 1 Processor -- start 2023-06-08T03:50:38.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.802+0000 7f056a020700 1 -- start start 2023-06-08T03:50:38.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f056a020700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f056415c1f0 0x7f056406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f056a020700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f05640cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f056a020700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05640cd140 con 0x7f05640cf130 2023-06-08T03:50:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f056a020700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0564071110 con 0x7f056415c1f0 2023-06-08T03:50:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f056a020700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0564071be0 con 0x7f0564071fe0 2023-06-08T03:50:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f056415c1f0 0x7f056406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f056415c1f0 0x7f056406d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34748/0 (socket says 172.21.15.79:34748) 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 -- 172.21.15.79:0/1291441404 learned_addr learned my addr 172.21.15.79:0/1291441404 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f0562ffd700 1 --2- 172.21.15.79:0/1291441404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f05640cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 -- 172.21.15.79:0/1291441404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 msgr2=0x7f05640cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 --2- 172.21.15.79:0/1291441404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f05640cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 -- 172.21.15.79:0/1291441404 >> v1:172.21.15.156:6789/0 conn(0x7f05640cf130 legacy=0x7f0564069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 -- 172.21.15.79:0/1291441404 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640cd390 con 0x7f056415c1f0 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.803+0000 7f05637fe700 1 --2- 172.21.15.79:0/1291441404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f056415c1f0 0x7f056406d8a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f05580098f0 tx=0x7f0558009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836b2a145384b820 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f05627fc700 1 -- 172.21.15.79:0/1291441404 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011830 con 0x7f056415c1f0 2023-06-08T03:50:38.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f05627fc700 1 -- 172.21.15.79:0/1291441404 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0558011990 con 0x7f056415c1f0 2023-06-08T03:50:38.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f05627fc700 1 -- 172.21.15.79:0/1291441404 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011d20 con 0x7f056415c1f0 2023-06-08T03:50:38.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 -- 172.21.15.79:0/1291441404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f056415c1f0 msgr2=0x7f056406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 --2- 172.21.15.79:0/1291441404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f056415c1f0 0x7f056406d8a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f05580098f0 tx=0x7f0558009c00 comp rx=0 tx=0).stop 2023-06-08T03:50:38.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 -- 172.21.15.79:0/1291441404 shutdown_connections 2023-06-08T03:50:38.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 --2- 172.21.15.79:0/1291441404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f056415c1f0 0x7f056406d8a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 --2- 172.21.15.79:0/1291441404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f05640cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 -- 172.21.15.79:0/1291441404 >> 172.21.15.79:0/1291441404 conn(0x7f05640b1d30 msgr2=0x7f056406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 -- 172.21.15.79:0/1291441404 shutdown_connections 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.804+0000 7f056a020700 1 -- 172.21.15.79:0/1291441404 wait complete. 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.805+0000 7f056a020700 1 Processor -- start 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 -- start start 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f0564152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 0x7f05640c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056415c1f0 0x7f05640c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c78f0 con 0x7f05640cf130 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c8fb0 con 0x7f056415c1f0 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f056a020700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c9290 con 0x7f0564071fe0 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f0562ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 0x7f05640c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f0562ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 0x7f05640c5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34756/0 (socket says 172.21.15.79:34756) 2023-06-08T03:50:38.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f0562ffd700 1 -- 172.21.15.79:0/1634451465 learned_addr learned my addr 172.21.15.79:0/1634451465 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.806+0000 7f05637fe700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f0564152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0563fff700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056415c1f0 0x7f05640c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0562ffd700 1 -- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 msgr2=0x7f0564152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0562ffd700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f0564152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0562ffd700 1 -- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056415c1f0 msgr2=0x7f05640c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0562ffd700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056415c1f0 0x7f05640c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0562ffd700 1 -- 172.21.15.79:0/1634451465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0558009550 con 0x7f05640cf130 2023-06-08T03:50:38.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f0562ffd700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 0x7f05640c5750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f055400e870 tx=0x7f055400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554009590 con 0x7f05640cf130 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055400ee70 con 0x7f05640cf130 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05640c9570 con 0x7f05640cf130 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055400ca50 con 0x7f05640cf130 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.807+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05640c7e70 con 0x7f05640cf130 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.808+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05640c81e0 con 0x7f05640cf130 2023-06-08T03:50:38.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.808+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f055400cbb0 con 0x7f05640cf130 2023-06-08T03:50:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.809+0000 7f054bfff700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f055006ace0 0x7f055006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.809+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0554089740 con 0x7f05640cf130 2023-06-08T03:50:38.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.809+0000 7f05637fe700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f055006ace0 0x7f055006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.809+0000 7f05637fe700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f055006ace0 0x7f055006d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0558000c00 tx=0x7f0558008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:38.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:38.813+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0554057f90 con 0x7f05640cf130 2023-06-08T03:50:39.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.017+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f05640c8000 con 0x7f05640cf130 2023-06-08T03:50:39.021 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.017+0000 7f054bfff700 1 -- 172.21.15.79:0/1634451465 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f055400ce60 con 0x7f05640cf130 2023-06-08T03:50:39.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f055006ace0 msgr2=0x7f055006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f055006ace0 0x7f055006d190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0558000c00 tx=0x7f0558008030 comp rx=0 tx=0).stop 2023-06-08T03:50:39.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 msgr2=0x7f05640c5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 0x7f05640c5750 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f055400e870 tx=0x7f055400ec30 comp rx=0 tx=0).stop 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 shutdown_connections 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f056415c1f0 0x7f05640c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f055006ace0 0x7f055006d190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0564071fe0 0x7f0564152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 --2- 172.21.15.79:0/1634451465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05640cf130 0x7f05640c5750 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 >> 172.21.15.79:0/1634451465 conn(0x7f05640b1d30 msgr2=0x7f056415d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 shutdown_connections 2023-06-08T03:50:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.019+0000 7f056a020700 1 -- 172.21.15.79:0/1634451465 wait complete. 2023-06-08T03:50:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:50:39.035 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2023-06-08T03:50:39.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:39.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-06-08T03:50:39.072 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[107289]: fuse finished with error 0 and tester_r 0 2023-06-08T03:50:39.073 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:50:39.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:50:39.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:39.141 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-06-08T03:50:39.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:39.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.331+0000 7f07200fb700 1 Processor -- start 2023-06-08T03:50:39.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.331+0000 7f07200fb700 1 -- start start 2023-06-08T03:50:39.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f07200fb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f07180b1650 0x7f07180ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f07200fb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07180cea60 con 0x7f071806b0d0 2023-06-08T03:50:39.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f07200fb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07180ced40 con 0x7f0718160080 2023-06-08T03:50:39.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f07200fb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180cf6e0 con 0x7f07180b1650 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071de97700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f071806b0d0 0x7f071815f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58238/0 (socket says 172.21.15.79:58238) 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071de97700 1 -- 172.21.15.79:0/2091697938 learned_addr learned my addr 172.21.15.79:0/2091697938 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889835920 0 0) 0x7f07180cea60 con 0x7f071806b0d0 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07040035b0 con 0x7f071806b0d0 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071d696700 1 --2- 172.21.15.79:0/2091697938 >> v2:172.21.15.156:3300/0 conn(0x7f07180b1650 0x7f07180ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2564724249 0 0) 0x7f07040035b0 con 0x7f071806b0d0 2023-06-08T03:50:39.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 >> v2:172.21.15.156:3300/0 conn(0x7f07180b1650 msgr2=0x7f07180ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 --2- 172.21.15.79:0/2091697938 >> v2:172.21.15.156:3300/0 conn(0x7f07180b1650 0x7f07180ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 >> v1:172.21.15.156:6789/0 conn(0x7f0718160080 legacy=0x7f07180cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180cf020 con 0x7f071806b0d0 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f0708001670 con 0x7f071806b0d0 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071d696700 1 --2- 172.21.15.79:0/2091697938 >> v2:172.21.15.156:3300/0 conn(0x7f07180b1650 0x7f07180ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 >> v1:172.21.15.79:6789/0 conn(0x7f071806b0d0 legacy=0x7f071815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704003a70 0x7f0704003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.332+0000 7f071ce95700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0704004850 0x7f0704004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0704005270 0x7f070400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180cf6e0 con 0x7f0704005270 2023-06-08T03:50:39.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070400c1b0 con 0x7f0704003a70 2023-06-08T03:50:39.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070400c370 con 0x7f0704004850 2023-06-08T03:50:39.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071de97700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704003a70 0x7f0704003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0704005270 0x7f070400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071d696700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0704004850 0x7f0704004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 -- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0704004850 msgr2=0x7f0704004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0704004850 0x7f0704004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 -- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704003a70 msgr2=0x7f0704003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704003a70 0x7f0704003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 -- 172.21.15.79:0/2091697938 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07040035b0 con 0x7f0704005270 2023-06-08T03:50:39.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 -- 172.21.15.79:0/2091697938 reap_dead start 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071e698700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0704005270 0x7f070400ba10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07181604a0 tx=0x7f0704005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8905592f58312cad server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710001db0 con 0x7f0704005270 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0710003c00 con 0x7f0704005270 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.333+0000 7f071ce95700 1 -- 172.21.15.79:0/2091697938 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000c980 con 0x7f0704005270 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 -- 172.21.15.79:0/2091697938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0704005270 msgr2=0x7f070400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0704005270 0x7f070400ba10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07181604a0 tx=0x7f0704005880 comp rx=0 tx=0).stop 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 -- 172.21.15.79:0/2091697938 shutdown_connections 2023-06-08T03:50:39.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 --2- 172.21.15.79:0/2091697938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0704005270 0x7f070400ba10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 -- 172.21.15.79:0/2091697938 >> 172.21.15.79:0/2091697938 conn(0x7f071806a670 msgr2=0x7f0718099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:39.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 -- 172.21.15.79:0/2091697938 shutdown_connections 2023-06-08T03:50:39.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.334+0000 7f07200fb700 1 -- 172.21.15.79:0/2091697938 wait complete. 2023-06-08T03:50:39.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.335+0000 7f07200fb700 1 Processor -- start 2023-06-08T03:50:39.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.335+0000 7f07200fb700 1 -- start start 2023-06-08T03:50:39.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704005270 0x7f07180afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07181593b0 0x7f07181f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 0x7f07181f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180af740 con 0x7f07180b5f70 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07181f5b90 con 0x7f0704005270 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07181f6080 con 0x7f07181593b0 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 0x7f07181f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 0x7f07181f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34778/0 (socket says 172.21.15.79:34778) 2023-06-08T03:50:39.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 -- 172.21.15.79:0/2336332029 learned_addr learned my addr 172.21.15.79:0/2336332029 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:39.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071de97700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704005270 0x7f07180afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071d696700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07181593b0 0x7f07181f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 -- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07181593b0 msgr2=0x7f07181f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07181593b0 0x7f07181f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 -- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704005270 msgr2=0x7f07180afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704005270 0x7f07180afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 -- 172.21.15.79:0/2336332029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07181f5de0 con 0x7f07180b5f70 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071d696700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07181593b0 0x7f07181f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f071e698700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 0x7f07181f5650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0710009000 tx=0x7f07100014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000ce10 con 0x7f07180b5f70 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0710005070 con 0x7f07180b5f70 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07180c6d00 con 0x7f07180b5f70 2023-06-08T03:50:39.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710005420 con 0x7f07180b5f70 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.336+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07180ad5f0 con 0x7f07180b5f70 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.337+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0718073e90 con 0x7f07180b5f70 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.338+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0710005580 con 0x7f07180b5f70 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.338+0000 7f070e7fc700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f06fc06ace0 0x7f06fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.338+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f071008a930 con 0x7f07180b5f70 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.338+0000 7f071de97700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f06fc06ace0 0x7f06fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.339+0000 7f071de97700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f06fc06ace0 0x7f06fc06d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07080018e0 tx=0x7f0708000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.342+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f07100558d0 con 0x7f07180b5f70 2023-06-08T03:50:39.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.528+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0718072ec0 con 0x7f07180b5f70 2023-06-08T03:50:39.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.529+0000 7f070e7fc700 1 -- 172.21.15.79:0/2336332029 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f0710016090 con 0x7f07180b5f70 2023-06-08T03:50:39.533 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:39.533 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:39.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f06fc06ace0 msgr2=0x7f06fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f06fc06ace0 0x7f06fc06d190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f07080018e0 tx=0x7f0708000bc0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 msgr2=0x7f07181f5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 0x7f07181f5650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0710009000 tx=0x7f07100014a0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 shutdown_connections 2023-06-08T03:50:39.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07180b5f70 0x7f07181f5650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0704005270 0x7f07180afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f06fc06ace0 0x7f06fc06d190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 --2- 172.21.15.79:0/2336332029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07181593b0 0x7f07181f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 >> 172.21.15.79:0/2336332029 conn(0x7f071806a670 msgr2=0x7f07180726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:39.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 shutdown_connections 2023-06-08T03:50:39.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.531+0000 7f07200fb700 1 -- 172.21.15.79:0/2336332029 wait complete. 2023-06-08T03:50:39.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:39.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.662+0000 7f531c46b700 1 Processor -- start 2023-06-08T03:50:39.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.662+0000 7f531c46b700 1 -- start start 2023-06-08T03:50:39.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531c46b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f531406a460 0x7f5314071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531c46b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53140cbb10 con 0x7f5314071a30 2023-06-08T03:50:39.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531c46b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53140cbdf0 con 0x7f531415a2f0 2023-06-08T03:50:39.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531c46b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53140cc0d0 con 0x7f531406a460 2023-06-08T03:50:39.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531aa08700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f531406a460 0x7f5314071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531aa08700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f531406a460 0x7f5314071ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34786/0 (socket says 172.21.15.79:34786) 2023-06-08T03:50:39.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f5319a06700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5314071a30 0x7f5314069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58260/0 (socket says 172.21.15.79:58260) 2023-06-08T03:50:39.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531aa08700 1 -- 172.21.15.79:0/156614183 learned_addr learned my addr 172.21.15.79:0/156614183 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:39.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531aa08700 1 -- 172.21.15.79:0/156614183 >> v1:172.21.15.156:6789/0 conn(0x7f531415a2f0 legacy=0x7f53140cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531aa08700 1 -- 172.21.15.79:0/156614183 >> v1:172.21.15.79:6789/0 conn(0x7f5314071a30 legacy=0x7f5314069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f5319205700 1 -- 172.21.15.79:0/156614183 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693899425 0 0) 0x7f53140cbb10 con 0x7f5314071a30 2023-06-08T03:50:39.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.663+0000 7f531aa08700 1 -- 172.21.15.79:0/156614183 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140cc3b0 con 0x7f531406a460 2023-06-08T03:50:39.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531aa08700 1 --2- 172.21.15.79:0/156614183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f531406a460 0x7f5314071ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f530c009e50 tx=0x7f530c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f398e2d5a6489e38 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:39.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f5319205700 1 -- 172.21.15.79:0/156614183 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00e070 con 0x7f531406a460 2023-06-08T03:50:39.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f5319205700 1 -- 172.21.15.79:0/156614183 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00b9c0 con 0x7f531406a460 2023-06-08T03:50:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f5319205700 1 -- 172.21.15.79:0/156614183 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00bd70 con 0x7f531406a460 2023-06-08T03:50:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 -- 172.21.15.79:0/156614183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f531406a460 msgr2=0x7f5314071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 --2- 172.21.15.79:0/156614183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f531406a460 0x7f5314071ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f530c009e50 tx=0x7f530c00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 -- 172.21.15.79:0/156614183 shutdown_connections 2023-06-08T03:50:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 --2- 172.21.15.79:0/156614183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f531406a460 0x7f5314071ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 -- 172.21.15.79:0/156614183 >> 172.21.15.79:0/156614183 conn(0x7f53140b1d20 msgr2=0x7f531406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 -- 172.21.15.79:0/156614183 shutdown_connections 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.664+0000 7f531c46b700 1 -- 172.21.15.79:0/156614183 wait complete. 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.665+0000 7f531c46b700 1 Processor -- start 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.665+0000 7f531c46b700 1 -- start start 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f531c46b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f531406a460 0x7f53140cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f531c46b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 0x7f5314155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f531c46b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531415a2f0 0x7f53141561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f531c46b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314155ae0 con 0x7f5314071a30 2023-06-08T03:50:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f531c46b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53140c85a0 con 0x7f531406a460 2023-06-08T03:50:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f531c46b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53140c8880 con 0x7f531415a2f0 2023-06-08T03:50:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f5319a06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 0x7f5314155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f5319a06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 0x7f5314155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34790/0 (socket says 172.21.15.79:34790) 2023-06-08T03:50:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.666+0000 7f5319a06700 1 -- 172.21.15.79:0/822824913 learned_addr learned my addr 172.21.15.79:0/822824913 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f531a207700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f531406a460 0x7f53140cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f5319a06700 1 -- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531415a2f0 msgr2=0x7f53141561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f5319a06700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531415a2f0 0x7f53141561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f5319a06700 1 -- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f531406a460 msgr2=0x7f53140cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f5319a06700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f531406a460 0x7f53140cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f5319a06700 1 -- 172.21.15.79:0/822824913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c009b30 con 0x7f5314071a30 2023-06-08T03:50:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f531a207700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f531406a460 0x7f53140cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:39.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f5319a06700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 0x7f5314155cb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f531000c520 tx=0x7f531000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:39.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310004c80 con 0x7f5314071a30 2023-06-08T03:50:39.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310004e00 con 0x7f5314071a30 2023-06-08T03:50:39.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53140c8b60 con 0x7f5314071a30 2023-06-08T03:50:39.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310007310 con 0x7f5314071a30 2023-06-08T03:50:39.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.667+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53140ce2f0 con 0x7f5314071a30 2023-06-08T03:50:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.668+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53140c9080 con 0x7f5314071a30 2023-06-08T03:50:39.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.668+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5310007470 con 0x7f5314071a30 2023-06-08T03:50:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.669+0000 7f530affd700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f52f806ace0 0x7f52f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.669+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5310089410 con 0x7f5314071a30 2023-06-08T03:50:39.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.669+0000 7f531a207700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f52f806ace0 0x7f52f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:39.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.669+0000 7f531a207700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f52f806ace0 0x7f52f806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5304000f60 tx=0x7f5304009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:39.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.673+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f53140c9080 con 0x7f5314071a30 2023-06-08T03:50:39.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.859+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53140c9080 con 0x7f5314071a30 2023-06-08T03:50:39.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.860+0000 7f530affd700 1 -- 172.21.15.79:0/822824913 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f53140c9080 con 0x7f5314071a30 2023-06-08T03:50:39.864 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:39.865 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:39.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f52f806ace0 msgr2=0x7f52f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f52f806ace0 0x7f52f806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5304000f60 tx=0x7f5304009040 comp rx=0 tx=0).stop 2023-06-08T03:50:39.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 msgr2=0x7f5314155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:39.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 0x7f5314155cb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f531000c520 tx=0x7f531000c8e0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 shutdown_connections 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f52f806ace0 0x7f52f806d190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f531415a2f0 0x7f53141561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f531406a460 0x7f53140cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 --2- 172.21.15.79:0/822824913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5314071a30 0x7f5314155cb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 >> 172.21.15.79:0/822824913 conn(0x7f53140b1d20 msgr2=0x7f5314069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 shutdown_connections 2023-06-08T03:50:39.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.862+0000 7f531c46b700 1 -- 172.21.15.79:0/822824913 wait complete. 2023-06-08T03:50:39.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-08T03:50:40.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.998+0000 7faef07ac700 1 Processor -- start 2023-06-08T03:50:40.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.998+0000 7faef07ac700 1 -- start start 2023-06-08T03:50:40.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.999+0000 7faef07ac700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 0x7faee80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.999+0000 7faef07ac700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80cae50 con 0x7faee80cf490 2023-06-08T03:50:40.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.999+0000 7faef07ac700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80b2f10 con 0x7faee80cd080 2023-06-08T03:50:40.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.999+0000 7faef07ac700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80cbea0 con 0x7faee80d0060 2023-06-08T03:50:40.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.999+0000 7faeeed49700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7faee80cf490 0x7faee80b2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58274/0 (socket says 172.21.15.79:58274) 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:39.999+0000 7faeeed49700 1 -- 172.21.15.79:0/2706566081 learned_addr learned my addr 172.21.15.79:0/2706566081 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeee548700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 0x7faee80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137491783 0 0) 0x7faee80cae50 con 0x7faee80cf490 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed40035b0 con 0x7faee80cf490 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120160748 0 0) 0x7faee80b2f10 con 0x7faee80cd080 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee80cae50 con 0x7faee80cd080 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1551275747 0 0) 0x7faed40035b0 con 0x7faee80cf490 2023-06-08T03:50:40.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 msgr2=0x7faee80af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 0x7faee80af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 >> v1:172.21.15.156:6789/0 conn(0x7faee80cd080 legacy=0x7faee80b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee815d930 con 0x7faee80cf490 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7faee00019f0 con 0x7faee80cf490 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 >> v1:172.21.15.79:6789/0 conn(0x7faee80cf490 legacy=0x7faee80b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 0x7faed4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004120 0x7faed4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faed4004d90 0x7faed4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed400bac0 con 0x7faed4004d90 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed400bc00 con 0x7faed4004120 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed400bdc0 con 0x7faee80d0060 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeeed49700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faed4004d90 0x7faed4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeee548700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 0x7faed4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.000+0000 7faeedd47700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004120 0x7faed4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 -- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 msgr2=0x7faed4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80d0060 0x7faed4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 -- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004120 msgr2=0x7faed4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004120 0x7faed4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 -- 172.21.15.79:0/2706566081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed40035b0 con 0x7faed4004d90 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faed4004d90 0x7faed4005200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faee0002230 tx=0x7faed4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90398f5da502a902 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0012600 con 0x7faed4004d90 2023-06-08T03:50:40.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee0012780 con 0x7faed4004d90 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeed546700 1 -- 172.21.15.79:0/2706566081 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0012b30 con 0x7faed4004d90 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faef07ac700 1 -- 172.21.15.79:0/2706566081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faed4004d90 msgr2=0x7faed4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faef07ac700 1 --2- 172.21.15.79:0/2706566081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faed4004d90 0x7faed4005200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faee0002230 tx=0x7faed4005880 comp rx=0 tx=0).stop 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faeeed49700 1 -- 172.21.15.79:0/2706566081 reap_dead start 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faef07ac700 1 -- 172.21.15.79:0/2706566081 shutdown_connections 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faef07ac700 1 -- 172.21.15.79:0/2706566081 >> 172.21.15.79:0/2706566081 conn(0x7faee806e1e0 msgr2=0x7faee806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faef07ac700 1 -- 172.21.15.79:0/2706566081 shutdown_connections 2023-06-08T03:50:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.001+0000 7faef07ac700 1 -- 172.21.15.79:0/2706566081 wait complete. 2023-06-08T03:50:40.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 Processor -- start 2023-06-08T03:50:40.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 -- start start 2023-06-08T03:50:40.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004d90 0x7faee8149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80a9df0 0x7faee80aa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 0x7faee80c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee80c6b80 con 0x7faee80c5b70 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee814d1f0 con 0x7faed4004d90 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faef07ac700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80aa7a0 con 0x7faee80a9df0 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.003+0000 7faeeed49700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 0x7faee80c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 0x7faee80c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34810/0 (socket says 172.21.15.79:34810) 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 -- 172.21.15.79:0/3986465955 learned_addr learned my addr 172.21.15.79:0/3986465955 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeee548700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004d90 0x7faee8149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeedd47700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80a9df0 0x7faee80aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 -- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80a9df0 msgr2=0x7faee80aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80a9df0 0x7faee80aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 -- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004d90 msgr2=0x7faee8149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004d90 0x7faee8149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 -- 172.21.15.79:0/3986465955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80c68a0 con 0x7faee80c5b70 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeedd47700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80a9df0 0x7faee80aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faeeed49700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 0x7faee80c5ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faee0015040 tx=0x7faee0001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee00019b0 con 0x7faee80c5b70 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee0012610 con 0x7faee80c5b70 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee001e620 con 0x7faee80c5b70 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee8149ea0 con 0x7faee80c5b70 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.004+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee814a460 con 0x7faee80c5b70 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.005+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee8149d00 con 0x7faee80c5b70 2023-06-08T03:50:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.005+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faee0019030 con 0x7faee80c5b70 2023-06-08T03:50:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.006+0000 7faedeffd700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faecc06ace0 0x7faecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.006+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faee008ae70 con 0x7faee80c5b70 2023-06-08T03:50:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.006+0000 7faeee548700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faecc06ace0 0x7faecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.006+0000 7faeee548700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faecc06ace0 0x7faecc06d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faed8009c60 tx=0x7faed8009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.010+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faee8149d00 con 0x7faee80c5b70 2023-06-08T03:50:40.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.242+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faee8149d00 con 0x7faee80c5b70 2023-06-08T03:50:40.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.942+0000 7faedeffd700 1 -- 172.21.15.79:0/3986465955 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7faee8149d00 con 0x7faee80c5b70 2023-06-08T03:50:40.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faecc06ace0 msgr2=0x7faecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faecc06ace0 0x7faecc06d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faed8009c60 tx=0x7faed8009210 comp rx=0 tx=0).stop 2023-06-08T03:50:40.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 msgr2=0x7faee80c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:40.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 0x7faee80c5ff0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faee0015040 tx=0x7faee0001d00 comp rx=0 tx=0).stop 2023-06-08T03:50:40.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 shutdown_connections 2023-06-08T03:50:40.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faee80c5b70 0x7faee80c5ff0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faed4004d90 0x7faee8149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faecc06ace0 0x7faecc06d190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 --2- 172.21.15.79:0/3986465955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faee80a9df0 0x7faee80aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:40.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 >> 172.21.15.79:0/3986465955 conn(0x7faee806e1e0 msgr2=0x7faee80b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:40.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 shutdown_connections 2023-06-08T03:50:40.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:40.945+0000 7faef07ac700 1 -- 172.21.15.79:0/3986465955 wait complete. 2023-06-08T03:50:40.963 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-06-08T03:50:40.964 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-08T03:50:41.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.087+0000 7fc2ef95a700 1 Processor -- start 2023-06-08T03:50:41.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.087+0000 7fc2ef95a700 1 -- start start 2023-06-08T03:50:41.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ef95a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2e815c1f0 0x7fc2e806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:41.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ef95a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc2e8071fe0 0x7fc2e80cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:41.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ef95a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e80cd140 con 0x7fc2e80cf130 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ef95a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e8071110 con 0x7fc2e815c1f0 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ef95a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e8071be0 con 0x7fc2e8071fe0 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ed6f6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2e815c1f0 0x7fc2e806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2edef7700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc2e80cf130 0x7fc2e8069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58290/0 (socket says 172.21.15.79:58290) 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2edef7700 1 -- 172.21.15.79:0/3208229704 learned_addr learned my addr 172.21.15.79:0/3208229704 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ecef5700 1 --2- 172.21.15.79:0/3208229704 >> v2:172.21.15.156:3300/0 conn(0x7fc2e8071fe0 0x7fc2e80cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ed6f6700 1 -- 172.21.15.79:0/3208229704 >> v2:172.21.15.156:3300/0 conn(0x7fc2e8071fe0 msgr2=0x7fc2e80cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ed6f6700 1 --2- 172.21.15.79:0/3208229704 >> v2:172.21.15.156:3300/0 conn(0x7fc2e8071fe0 0x7fc2e80cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ed6f6700 1 -- 172.21.15.79:0/3208229704 >> v1:172.21.15.79:6789/0 conn(0x7fc2e80cf130 legacy=0x7fc2e8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ed6f6700 1 -- 172.21.15.79:0/3208229704 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80cd390 con 0x7fc2e815c1f0 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2dffff700 1 -- 172.21.15.79:0/3208229704 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532527061 0 0) 0x7fc2e80cd140 con 0x7fc2e80cf130 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.088+0000 7fc2ecef5700 1 --2- 172.21.15.79:0/3208229704 >> v2:172.21.15.156:3300/0 conn(0x7fc2e8071fe0 0x7fc2e80cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ed6f6700 1 --2- 172.21.15.79:0/3208229704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e815c1f0 0x7fc2e806d8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80d0670 tx=0x7fc2d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de15cdc35aa7003 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2dffff700 1 -- 172.21.15.79:0/3208229704 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8011830 con 0x7fc2e815c1f0 2023-06-08T03:50:41.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2dffff700 1 -- 172.21.15.79:0/3208229704 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d8011990 con 0x7fc2e815c1f0 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2dffff700 1 -- 172.21.15.79:0/3208229704 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8011d20 con 0x7fc2e815c1f0 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 -- 172.21.15.79:0/3208229704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e815c1f0 msgr2=0x7fc2e806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/3208229704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e815c1f0 0x7fc2e806d8a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80d0670 tx=0x7fc2d8009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 -- 172.21.15.79:0/3208229704 shutdown_connections 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/3208229704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e815c1f0 0x7fc2e806d8a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/3208229704 >> v2:172.21.15.156:3300/0 conn(0x7fc2e8071fe0 0x7fc2e80cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 -- 172.21.15.79:0/3208229704 >> 172.21.15.79:0/3208229704 conn(0x7fc2e80b1d30 msgr2=0x7fc2e806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 -- 172.21.15.79:0/3208229704 shutdown_connections 2023-06-08T03:50:41.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.089+0000 7fc2ef95a700 1 -- 172.21.15.79:0/3208229704 wait complete. 2023-06-08T03:50:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.090+0000 7fc2ef95a700 1 Processor -- start 2023-06-08T03:50:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 -- start start 2023-06-08T03:50:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e8071fe0 0x7fc2e80b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 0x7fc2e81f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2e815c1f0 0x7fc2e80b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80b27e0 con 0x7fc2e80cf130 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80b2ac0 con 0x7fc2e8071fe0 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ef95a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80b2da0 con 0x7fc2e815c1f0 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ecef5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 0x7fc2e81f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ecef5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 0x7fc2e81f1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34822/0 (socket says 172.21.15.79:34822) 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ecef5700 1 -- 172.21.15.79:0/2312648609 learned_addr learned my addr 172.21.15.79:0/2312648609 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.091+0000 7fc2ed6f6700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e8071fe0 0x7fc2e80b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ecef5700 1 -- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2e815c1f0 msgr2=0x7fc2e80b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2edef7700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2e815c1f0 0x7fc2e80b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ecef5700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2e815c1f0 0x7fc2e80b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ecef5700 1 -- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e8071fe0 msgr2=0x7fc2e80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ecef5700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e8071fe0 0x7fc2e80b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ecef5700 1 -- 172.21.15.79:0/2312648609 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80094a0 con 0x7fc2e80cf130 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ed6f6700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e8071fe0 0x7fc2e80b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2edef7700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2e815c1f0 0x7fc2e80b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ecef5700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 0x7fc2e81f1010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c7f0 tx=0x7fc2e400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4011800 con 0x7fc2e80cf130 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e4011960 con 0x7fc2e80cf130 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e80b3080 con 0x7fc2e80cf130 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4011d10 con 0x7fc2e80cf130 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.092+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e80b3dc0 con 0x7fc2e80cf130 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.093+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e80b35a0 con 0x7fc2e80cf130 2023-06-08T03:50:41.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.093+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2e4018360 con 0x7fc2e80cf130 2023-06-08T03:50:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.094+0000 7fc2ddffb700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2cc06ace0 0x7fc2cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.094+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc2e4089710 con 0x7fc2e80cf130 2023-06-08T03:50:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.094+0000 7fc2ed6f6700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2cc06ace0 0x7fc2cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.095+0000 7fc2ed6f6700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2cc06ace0 0x7fc2cc06d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009b60 tx=0x7fc2d8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:41.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.098+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc2e80b35a0 con 0x7fc2e80cf130 2023-06-08T03:50:41.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.329+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc2e80b35a0 con 0x7fc2e80cf130 2023-06-08T03:50:41.954 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.950+0000 7fc2ddffb700 1 -- 172.21.15.79:0/2312648609 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2e80b35a0 con 0x7fc2e80cf130 2023-06-08T03:50:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2cc06ace0 msgr2=0x7fc2cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2cc06ace0 0x7fc2cc06d190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009b60 tx=0x7fc2d8008030 comp rx=0 tx=0).stop 2023-06-08T03:50:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 msgr2=0x7fc2e81f1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 0x7fc2e81f1010 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c7f0 tx=0x7fc2e400cbb0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 shutdown_connections 2023-06-08T03:50:41.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2cc06ace0 0x7fc2cc06d190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2e815c1f0 0x7fc2e80b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2e8071fe0 0x7fc2e80b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 --2- 172.21.15.79:0/2312648609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2e80cf130 0x7fc2e81f1010 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.952+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 >> 172.21.15.79:0/2312648609 conn(0x7fc2e80b1d30 msgr2=0x7fc2e80cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.953+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 shutdown_connections 2023-06-08T03:50:41.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:41.953+0000 7fc2ef95a700 1 -- 172.21.15.79:0/2312648609 wait complete. 2023-06-08T03:50:41.969 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-08T03:50:41.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:42.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.086+0000 7ff1846ff700 1 Processor -- start 2023-06-08T03:50:42.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.086+0000 7ff1846ff700 1 -- start start 2023-06-08T03:50:42.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.087+0000 7ff1846ff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff17c06b0d0 0x7ff17c0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.087+0000 7ff1846ff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff17c160080 0x7ff17c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.087+0000 7ff1846ff700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c0cea60 con 0x7ff17c0b1650 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.087+0000 7ff1846ff700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0cf6e0 con 0x7ff17c06b0d0 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.087+0000 7ff1846ff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c0ced40 con 0x7ff17c160080 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff18249b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff17c06b0d0 0x7ff17c0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff181c9a700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff17c0b1650 0x7ff17c15f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58304/0 (socket says 172.21.15.79:58304) 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff181c9a700 1 -- 172.21.15.79:0/2681056639 learned_addr learned my addr 172.21.15.79:0/2681056639 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff182c9c700 1 --2- 172.21.15.79:0/2681056639 >> v2:172.21.15.156:3300/0 conn(0x7ff17c160080 0x7ff17c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff18249b700 1 -- 172.21.15.79:0/2681056639 >> v2:172.21.15.156:3300/0 conn(0x7ff17c160080 msgr2=0x7ff17c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff18249b700 1 --2- 172.21.15.79:0/2681056639 >> v2:172.21.15.156:3300/0 conn(0x7ff17c160080 0x7ff17c0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff18249b700 1 -- 172.21.15.79:0/2681056639 >> v1:172.21.15.79:6789/0 conn(0x7ff17c0b1650 legacy=0x7ff17c15f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff18249b700 1 -- 172.21.15.79:0/2681056639 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0cf020 con 0x7ff17c06b0d0 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff181499700 1 -- 172.21.15.79:0/2681056639 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098342383 0 0) 0x7ff17c0cea60 con 0x7ff17c0b1650 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff18249b700 1 --2- 172.21.15.79:0/2681056639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c06b0d0 0x7ff17c0ce460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0c7820 tx=0x7ff16c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd9a893222ea5d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff181499700 1 -- 172.21.15.79:0/2681056639 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c011830 con 0x7ff17c06b0d0 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff181499700 1 -- 172.21.15.79:0/2681056639 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c011990 con 0x7ff17c06b0d0 2023-06-08T03:50:42.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.088+0000 7ff181499700 1 -- 172.21.15.79:0/2681056639 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c011d20 con 0x7ff17c06b0d0 2023-06-08T03:50:42.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 -- 172.21.15.79:0/2681056639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c06b0d0 msgr2=0x7ff17c0ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 --2- 172.21.15.79:0/2681056639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c06b0d0 0x7ff17c0ce460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0c7820 tx=0x7ff16c009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:42.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 -- 172.21.15.79:0/2681056639 shutdown_connections 2023-06-08T03:50:42.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 --2- 172.21.15.79:0/2681056639 >> v2:172.21.15.156:3300/0 conn(0x7ff17c160080 0x7ff17c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 --2- 172.21.15.79:0/2681056639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c06b0d0 0x7ff17c0ce460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 -- 172.21.15.79:0/2681056639 >> 172.21.15.79:0/2681056639 conn(0x7ff17c06a670 msgr2=0x7ff17c099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 -- 172.21.15.79:0/2681056639 shutdown_connections 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.089+0000 7ff1846ff700 1 -- 172.21.15.79:0/2681056639 wait complete. 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.090+0000 7ff1846ff700 1 Processor -- start 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.090+0000 7ff1846ff700 1 -- start start 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17c06b0d0 0x7ff17c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff17c0b1650 0x7ff17c156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 0x7ff17c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c0ae070 con 0x7ff17c160080 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1f67c0 con 0x7ff17c06b0d0 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c1566d0 con 0x7ff17c0b1650 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 0x7ff17c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 0x7ff17c15cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34836/0 (socket says 172.21.15.79:34836) 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 -- 172.21.15.79:0/575336480 learned_addr learned my addr 172.21.15.79:0/575336480 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 -- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff17c0b1650 msgr2=0x7ff17c156190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:42.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff17c0b1650 0x7ff17c156190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 -- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17c06b0d0 msgr2=0x7ff17c1f6280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17c06b0d0 0x7ff17c1f6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 -- 172.21.15.79:0/575336480 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0094a0 con 0x7ff17c160080 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff182c9c700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 0x7ff17c15cbc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff17400cd00 tx=0x7ff17400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174003a20 con 0x7ff17c160080 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174003ba0 con 0x7ff17c160080 2023-06-08T03:50:42.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c0ad3f0 con 0x7ff17c160080 2023-06-08T03:50:42.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174004020 con 0x7ff17c160080 2023-06-08T03:50:42.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.091+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c0ad9b0 con 0x7ff17c160080 2023-06-08T03:50:42.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.092+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c0ae1b0 con 0x7ff17c160080 2023-06-08T03:50:42.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.093+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff174004180 con 0x7ff17c160080 2023-06-08T03:50:42.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.093+0000 7ff172ffd700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff16006ace0 0x7ff16006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.093+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff174012070 con 0x7ff17c160080 2023-06-08T03:50:42.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.093+0000 7ff18249b700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff16006ace0 0x7ff16006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.094+0000 7ff18249b700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff16006ace0 0x7ff16006d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0090b0 tx=0x7ff16c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.097+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff17c0ae1b0 con 0x7ff17c160080 2023-06-08T03:50:42.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.329+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff17c0ae1b0 con 0x7ff17c160080 2023-06-08T03:50:42.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.329+0000 7ff172ffd700 1 -- 172.21.15.79:0/575336480 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7ff17c0ae1b0 con 0x7ff17c160080 2023-06-08T03:50:42.334 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:42.335 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:50:42.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff16006ace0 msgr2=0x7ff16006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff16006ace0 0x7ff16006d190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0090b0 tx=0x7ff16c008030 comp rx=0 tx=0).stop 2023-06-08T03:50:42.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 msgr2=0x7ff17c15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 0x7ff17c15cbc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff17400cd00 tx=0x7ff17400a570 comp rx=0 tx=0).stop 2023-06-08T03:50:42.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 shutdown_connections 2023-06-08T03:50:42.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff17c160080 0x7ff17c15cbc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff17c06b0d0 0x7ff17c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff16006ace0 0x7ff16006d190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 --2- 172.21.15.79:0/575336480 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff17c0b1650 0x7ff17c156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 >> 172.21.15.79:0/575336480 conn(0x7ff17c06a670 msgr2=0x7ff17c0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:42.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 shutdown_connections 2023-06-08T03:50:42.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.331+0000 7ff1846ff700 1 -- 172.21.15.79:0/575336480 wait complete. 2023-06-08T03:50:42.340 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:50:42.348 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:42.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.462+0000 7f8947a26700 1 Processor -- start 2023-06-08T03:50:42.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.463+0000 7f8947a26700 1 -- start start 2023-06-08T03:50:42.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.463+0000 7f8947a26700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 0x7f89400b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.463+0000 7f8947a26700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89400b0750 con 0x7f894015a640 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.463+0000 7f8947a26700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89400ceeb0 con 0x7f89400af580 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.463+0000 7f8947a26700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b09a0 con 0x7f894015aa10 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f89457c2700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f894015a640 0x7f89400ce3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58316/0 (socket says 172.21.15.79:58316) 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f89457c2700 1 -- 172.21.15.79:0/3114652507 learned_addr learned my addr 172.21.15.79:0/3114652507 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8944fc1700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 0x7f89400b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725305287 0 0) 0x7f89400b0750 con 0x7f894015a640 2023-06-08T03:50:42.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f892c0035b0 con 0x7f894015a640 2023-06-08T03:50:42.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 394952080 0 0) 0x7f892c0035b0 con 0x7f894015a640 2023-06-08T03:50:42.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 msgr2=0x7f89400b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:42.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 0x7f89400b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 >> v1:172.21.15.156:6789/0 conn(0x7f89400af580 legacy=0x7f89400ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400b0c80 con 0x7f894015a640 2023-06-08T03:50:42.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f8930001670 con 0x7f894015a640 2023-06-08T03:50:42.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 >> v1:172.21.15.79:6789/0 conn(0x7f894015a640 legacy=0x7f89400ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 0x7f892c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f892c003fe0 0x7f892c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f892c004d90 0x7f892c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b09a0 con 0x7f892c004d90 2023-06-08T03:50:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c00bac0 con 0x7f892c003fe0 2023-06-08T03:50:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c00bc80 con 0x7f894015aa10 2023-06-08T03:50:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8945fc3700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f892c004d90 0x7f892c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f89457c2700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 0x7f892c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8944fc1700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f892c003fe0 0x7f892c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8945fc3700 1 -- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 msgr2=0x7f892c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:42.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.464+0000 7f8945fc3700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f894015aa10 0x7f892c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8945fc3700 1 -- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f892c003fe0 msgr2=0x7f892c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:42.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8945fc3700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f892c003fe0 0x7f892c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8945fc3700 1 -- 172.21.15.79:0/3114652507 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0035b0 con 0x7f892c004d90 2023-06-08T03:50:42.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8945fc3700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f892c004d90 0x7f892c005200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f893800c260 tx=0x7f892c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9879b7b8dc352e6c server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938001e70 con 0x7f892c004d90 2023-06-08T03:50:42.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893800ce20 con 0x7f892c004d90 2023-06-08T03:50:42.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8937fff700 1 -- 172.21.15.79:0/3114652507 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938012660 con 0x7f892c004d90 2023-06-08T03:50:42.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8947a26700 1 -- 172.21.15.79:0/3114652507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f892c004d90 msgr2=0x7f892c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8947a26700 1 --2- 172.21.15.79:0/3114652507 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f892c004d90 0x7f892c005200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f893800c260 tx=0x7f892c005880 comp rx=0 tx=0).stop 2023-06-08T03:50:42.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8945fc3700 1 -- 172.21.15.79:0/3114652507 reap_dead start 2023-06-08T03:50:42.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8947a26700 1 -- 172.21.15.79:0/3114652507 shutdown_connections 2023-06-08T03:50:42.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8947a26700 1 -- 172.21.15.79:0/3114652507 >> 172.21.15.79:0/3114652507 conn(0x7f894006a670 msgr2=0x7f8940099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:42.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8947a26700 1 -- 172.21.15.79:0/3114652507 shutdown_connections 2023-06-08T03:50:42.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.465+0000 7f8947a26700 1 -- 172.21.15.79:0/3114652507 wait complete. 2023-06-08T03:50:42.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 Processor -- start 2023-06-08T03:50:42.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 -- start start 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400ca100 0x7f894006b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89400ca4d0 0x7f894006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 0x7f89401591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894006bf40 con 0x7f89400b4cc0 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b41e0 con 0x7f89400ca100 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8947a26700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b44c0 con 0x7f89400ca4d0 2023-06-08T03:50:42.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8945fc3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 0x7f89401591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8945fc3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 0x7f89401591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34854/0 (socket says 172.21.15.79:34854) 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.467+0000 7f8945fc3700 1 -- 172.21.15.79:0/1740507626 learned_addr learned my addr 172.21.15.79:0/1740507626 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f89457c2700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400ca100 0x7f894006b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8944fc1700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89400ca4d0 0x7f894006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8945fc3700 1 -- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89400ca4d0 msgr2=0x7f894006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8945fc3700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89400ca4d0 0x7f894006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8945fc3700 1 -- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400ca100 msgr2=0x7f894006b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8945fc3700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400ca100 0x7f894006b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8945fc3700 1 -- 172.21.15.79:0/1740507626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400b47a0 con 0x7f89400b4cc0 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8945fc3700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 0x7f89401591e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8938013000 tx=0x7f89380015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938003c80 con 0x7f89400b4cc0 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938020070 con 0x7f89400b4cc0 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938005300 con 0x7f89400b4cc0 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894006ce10 con 0x7f89400b4cc0 2023-06-08T03:50:42.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.468+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89400b5340 con 0x7f89400b4cc0 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.469+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89400b4030 con 0x7f89400b4cc0 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.470+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8938022030 con 0x7f89400b4cc0 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.470+0000 7f8935ffb700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f892406ace0 0x7f892406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.470+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f893808a120 con 0x7f89400b4cc0 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.470+0000 7f89457c2700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f892406ace0 0x7f892406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.471+0000 7f89457c2700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f892406ace0 0x7f892406d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89300048d0 tx=0x7f8930001170 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.474+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8938055150 con 0x7f89400b4cc0 2023-06-08T03:50:42.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.661+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8940073e90 con 0x7f89400b4cc0 2023-06-08T03:50:42.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.662+0000 7f8935ffb700 1 -- 172.21.15.79:0/1740507626 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f8938058770 con 0x7f89400b4cc0 2023-06-08T03:50:42.667 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:42.667 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f892406ace0 msgr2=0x7f892406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f892406ace0 0x7f892406d190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89300048d0 tx=0x7f8930001170 comp rx=0 tx=0).stop 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 msgr2=0x7f89401591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 0x7f89401591e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8938013000 tx=0x7f89380015b0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 shutdown_connections 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f89400b4cc0 0x7f89401591e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89400ca100 0x7f894006b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f892406ace0 0x7f892406d190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 --2- 172.21.15.79:0/1740507626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f89400ca4d0 0x7f894006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 >> 172.21.15.79:0/1740507626 conn(0x7f894006a670 msgr2=0x7f8940157f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:42.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 shutdown_connections 2023-06-08T03:50:42.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.665+0000 7f8947a26700 1 -- 172.21.15.79:0/1740507626 wait complete. 2023-06-08T03:50:42.681 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-08T03:50:42.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-08T03:50:42.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.796+0000 7fd8ae0d6700 1 Processor -- start 2023-06-08T03:50:42.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.796+0000 7fd8ae0d6700 1 -- start start 2023-06-08T03:50:42.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8ae0d6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8a8071a30 0x7fd8a8071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8ae0d6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80cbb10 con 0x7fd8a8069d50 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8ae0d6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80cbdf0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8ae0d6700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a80cc0d0 con 0x7fd8a8071a30 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a6ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8a8071a30 0x7fd8a8071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a7fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd8a8069d50 0x7fd8a806a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58338/0 (socket says 172.21.15.79:58338) 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a7fff700 1 -- 172.21.15.79:0/494042475 learned_addr learned my addr 172.21.15.79:0/494042475 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:42.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a6ffd700 1 -- 172.21.15.79:0/494042475 >> v1:172.21.15.156:6789/0 conn(0x7fd8a815a2f0 legacy=0x7fd8a80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a6ffd700 1 -- 172.21.15.79:0/494042475 >> v1:172.21.15.79:6789/0 conn(0x7fd8a8069d50 legacy=0x7fd8a806a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a6ffd700 1 -- 172.21.15.79:0/494042475 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80cc3b0 con 0x7fd8a8071a30 2023-06-08T03:50:42.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a6ffd700 1 --2- 172.21.15.79:0/494042475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a8071a30 0x7fd8a8071ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd898009840 tx=0x7fd898009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e018097313cc4c3e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.797+0000 7fd8a67fc700 1 -- 172.21.15.79:0/494042475 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898011830 con 0x7fd8a8071a30 2023-06-08T03:50:42.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8a67fc700 1 -- 172.21.15.79:0/494042475 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd898011990 con 0x7fd8a8071a30 2023-06-08T03:50:42.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8a67fc700 1 -- 172.21.15.79:0/494042475 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898011d20 con 0x7fd8a8071a30 2023-06-08T03:50:42.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/494042475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a8071a30 msgr2=0x7fd8a8071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/494042475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a8071a30 0x7fd8a8071ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd898009840 tx=0x7fd898009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/494042475 shutdown_connections 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/494042475 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a8071a30 0x7fd8a8071ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/494042475 >> 172.21.15.79:0/494042475 conn(0x7fd8a80b1d20 msgr2=0x7fd8a806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/494042475 shutdown_connections 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.798+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/494042475 wait complete. 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.799+0000 7fd8ae0d6700 1 Processor -- start 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.799+0000 7fd8ae0d6700 1 -- start start 2023-06-08T03:50:42.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8ae0d6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a8069d50 0x7fd8a8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8ae0d6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8071a30 0x7fd8a80c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8ae0d6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 0x7fd8a815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8ae0d6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c3380 con 0x7fd8a815a2f0 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8ae0d6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c7050 con 0x7fd8a8071a30 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8ae0d6700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80ae290 con 0x7fd8a8069d50 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 0x7fd8a815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 0x7fd8a815ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34866/0 (socket says 172.21.15.79:34866) 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a7fff700 1 -- 172.21.15.79:0/2027983542 learned_addr learned my addr 172.21.15.79:0/2027983542 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a77fe700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a8069d50 0x7fd8a8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a7fff700 1 -- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a8069d50 msgr2=0x7fd8a8154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a6ffd700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8071a30 0x7fd8a80c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.800+0000 7fd8a7fff700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a8069d50 0x7fd8a8154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd8a7fff700 1 -- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8071a30 msgr2=0x7fd8a80c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:42.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd8a7fff700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8071a30 0x7fd8a80c2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd8a7fff700 1 -- 172.21.15.79:0/2027983542 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8980094a0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd8a7fff700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 0x7fd8a815ca10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000ca80 tx=0x7fd8a000ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000aaa0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000ac00 con 0x7fd8a815a2f0 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a80c38b0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0012400 con 0x7fd8a815a2f0 2023-06-08T03:50:42.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.801+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a80c72a0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.802+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a80c74c0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.802+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8a001b3b0 con 0x7fd8a815a2f0 2023-06-08T03:50:42.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.803+0000 7fd88ffff700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd89406ace0 0x7fd89406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:42.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.803+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd8a0089670 con 0x7fd8a815a2f0 2023-06-08T03:50:42.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.803+0000 7fd8a77fe700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd89406ace0 0x7fd89406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:42.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.803+0000 7fd8a77fe700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd89406ace0 0x7fd89406d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000f60 tx=0x7fd89c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:42.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.807+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8a0054650 con 0x7fd8a815a2f0 2023-06-08T03:50:42.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:42.990+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 --> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8a80c9e30 con 0x7fd89406ace0 2023-06-08T03:50:43.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.001+0000 7fd88ffff700 1 -- 172.21.15.79:0/2027983542 <== mgr.4099 v2:172.21.15.156:6832/2537078349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367559 (secure 0 0 0) 0x7fd89c0050d0 con 0x7fd89406ace0 2023-06-08T03:50:43.009 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd89406ace0 msgr2=0x7fd89406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd89406ace0 0x7fd89406d190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd89c000f60 tx=0x7fd89c009040 comp rx=0 tx=0).stop 2023-06-08T03:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 msgr2=0x7fd8a815ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 0x7fd8a815ca10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000ca80 tx=0x7fd8a000ce40 comp rx=0 tx=0).stop 2023-06-08T03:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 shutdown_connections 2023-06-08T03:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a815a2f0 0x7fd8a815ca10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd89406ace0 0x7fd89406d190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a8069d50 0x7fd8a8154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 --2- 172.21.15.79:0/2027983542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a8071a30 0x7fd8a80c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 >> 172.21.15.79:0/2027983542 conn(0x7fd8a80b1d20 msgr2=0x7fd8a8159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 shutdown_connections 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.005+0000 7fd8ae0d6700 1 -- 172.21.15.79:0/2027983542 wait complete. 2023-06-08T03:50:43.012 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-08T03:50:43.021 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-06-08T03:50:41.873159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594819,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218920,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755800,"statfs":{"total":773094113280,"available":772869939200,"internally_reserved":0,"allocated":3383296,"data_stored":2064910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312366+0000","last_change":"2023-06-08T03:50:24.695168+0000","last_active":"2023-06-08T03:50:29.312366+0000","last_peered":"2023-06-08T03:50:29.312366+0000","last_clean":"2023-06-08T03:50:29.312366+0000","last_became_active":"2023-06-08T03:50:24.694872+0000","last_became_peered":"2023-06-08T03:50:24.694872+0000","last_unstale":"2023-06-08T03:50:29.312366+0000","last_undegraded":"2023-06-08T03:50:29.312366+0000","last_fullsized":"2023-06-08T03:50:29.312366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791005+0000","last_change":"2023-06-08T03:50:24.687043+0000","last_active":"2023-06-08T03:50:29.791005+0000","last_peered":"2023-06-08T03:50:29.791005+0000","last_clean":"2023-06-08T03:50:29.791005+0000","last_became_active":"2023-06-08T03:50:24.686710+0000","last_became_peered":"2023-06-08T03:50:24.686710+0000","last_unstale":"2023-06-08T03:50:29.791005+0000","last_undegraded":"2023-06-08T03:50:29.791005+0000","last_fullsized":"2023-06-08T03:50:29.791005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591260+0000","last_change":"2023-06-08T03:50:24.686759+0000","last_active":"2023-06-08T03:50:29.591260+0000","last_peered":"2023-06-08T03:50:29.591260+0000","last_clean":"2023-06-08T03:50:29.591260+0000","last_became_active":"2023-06-08T03:50:24.686497+0000","last_became_peered":"2023-06-08T03:50:24.686497+0000","last_unstale":"2023-06-08T03:50:29.591260+0000","last_undegraded":"2023-06-08T03:50:29.591260+0000","last_fullsized":"2023-06-08T03:50:29.591260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:14.651224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591320+0000","last_change":"2023-06-08T03:50:24.691468+0000","last_active":"2023-06-08T03:50:29.591320+0000","last_peered":"2023-06-08T03:50:29.591320+0000","last_clean":"2023-06-08T03:50:29.591320+0000","last_became_active":"2023-06-08T03:50:24.691218+0000","last_became_peered":"2023-06-08T03:50:24.691218+0000","last_unstale":"2023-06-08T03:50:29.591320+0000","last_undegraded":"2023-06-08T03:50:29.591320+0000","last_fullsized":"2023-06-08T03:50:29.591320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309624+0000","last_change":"2023-06-08T03:50:24.685604+0000","last_active":"2023-06-08T03:50:29.309624+0000","last_peered":"2023-06-08T03:50:29.309624+0000","last_clean":"2023-06-08T03:50:29.309624+0000","last_became_active":"2023-06-08T03:50:24.685184+0000","last_became_peered":"2023-06-08T03:50:24.685184+0000","last_unstale":"2023-06-08T03:50:29.309624+0000","last_undegraded":"2023-06-08T03:50:29.309624+0000","last_fullsized":"2023-06-08T03:50:29.309624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:09:46.870511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789772+0000","last_change":"2023-06-08T03:50:24.698386+0000","last_active":"2023-06-08T03:50:29.789772+0000","last_peered":"2023-06-08T03:50:29.789772+0000","last_clean":"2023-06-08T03:50:29.789772+0000","last_became_active":"2023-06-08T03:50:24.698086+0000","last_became_peered":"2023-06-08T03:50:24.698086+0000","last_unstale":"2023-06-08T03:50:29.789772+0000","last_undegraded":"2023-06-08T03:50:29.789772+0000","last_fullsized":"2023-06-08T03:50:29.789772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:11.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790111+0000","last_change":"2023-06-08T03:50:24.693046+0000","last_active":"2023-06-08T03:50:29.790111+0000","last_peered":"2023-06-08T03:50:29.790111+0000","last_clean":"2023-06-08T03:50:29.790111+0000","last_became_active":"2023-06-08T03:50:24.692581+0000","last_became_peered":"2023-06-08T03:50:24.692581+0000","last_unstale":"2023-06-08T03:50:29.790111+0000","last_undegraded":"2023-06-08T03:50:29.790111+0000","last_fullsized":"2023-06-08T03:50:29.790111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:39.113365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790425+0000","last_change":"2023-06-08T03:50:24.689352+0000","last_active":"2023-06-08T03:50:29.790425+0000","last_peered":"2023-06-08T03:50:29.790425+0000","last_clean":"2023-06-08T03:50:29.790425+0000","last_became_active":"2023-06-08T03:50:24.688961+0000","last_became_peered":"2023-06-08T03:50:24.688961+0000","last_unstale":"2023-06-08T03:50:29.790425+0000","last_undegraded":"2023-06-08T03:50:29.790425+0000","last_fullsized":"2023-06-08T03:50:29.790425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:59:48.614865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310084+0000","last_change":"2023-06-08T03:50:24.682755+0000","last_active":"2023-06-08T03:50:29.310084+0000","last_peered":"2023-06-08T03:50:29.310084+0000","last_clean":"2023-06-08T03:50:29.310084+0000","last_became_active":"2023-06-08T03:50:24.682417+0000","last_became_peered":"2023-06-08T03:50:24.682417+0000","last_unstale":"2023-06-08T03:50:29.310084+0000","last_undegraded":"2023-06-08T03:50:29.310084+0000","last_fullsized":"2023-06-08T03:50:29.310084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:38:17.580156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791038+0000","last_change":"2023-06-08T03:50:24.697828+0000","last_active":"2023-06-08T03:50:29.791038+0000","last_peered":"2023-06-08T03:50:29.791038+0000","last_clean":"2023-06-08T03:50:29.791038+0000","last_became_active":"2023-06-08T03:50:24.697576+0000","last_became_peered":"2023-06-08T03:50:24.697576+0000","last_unstale":"2023-06-08T03:50:29.791038+0000","last_undegraded":"2023-06-08T03:50:29.791038+0000","last_fullsized":"2023-06-08T03:50:29.791038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789439+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.789439+0000","last_peered":"2023-06-08T03:50:29.789439+0000","last_clean":"2023-06-08T03:50:29.789439+0000","last_became_active":"2023-06-08T03:50:24.688197+0000","last_became_peered":"2023-06-08T03:50:24.688197+0000","last_unstale":"2023-06-08T03:50:29.789439+0000","last_undegraded":"2023-06-08T03:50:29.789439+0000","last_fullsized":"2023-06-08T03:50:29.789439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:04:27.557616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311848+0000","last_change":"2023-06-08T03:50:24.692867+0000","last_active":"2023-06-08T03:50:29.311848+0000","last_peered":"2023-06-08T03:50:29.311848+0000","last_clean":"2023-06-08T03:50:29.311848+0000","last_became_active":"2023-06-08T03:50:24.692613+0000","last_became_peered":"2023-06-08T03:50:24.692613+0000","last_unstale":"2023-06-08T03:50:29.311848+0000","last_undegraded":"2023-06-08T03:50:29.311848+0000","last_fullsized":"2023-06-08T03:50:29.311848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:02.122906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309156+0000","last_change":"2023-06-08T03:50:24.694092+0000","last_active":"2023-06-08T03:50:29.309156+0000","last_peered":"2023-06-08T03:50:29.309156+0000","last_clean":"2023-06-08T03:50:29.309156+0000","last_became_active":"2023-06-08T03:50:24.693783+0000","last_became_peered":"2023-06-08T03:50:24.693783+0000","last_unstale":"2023-06-08T03:50:29.309156+0000","last_undegraded":"2023-06-08T03:50:29.309156+0000","last_fullsized":"2023-06-08T03:50:29.309156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790036+0000","last_change":"2023-06-08T03:50:24.702721+0000","last_active":"2023-06-08T03:50:29.790036+0000","last_peered":"2023-06-08T03:50:29.790036+0000","last_clean":"2023-06-08T03:50:29.790036+0000","last_became_active":"2023-06-08T03:50:24.702438+0000","last_became_peered":"2023-06-08T03:50:24.702438+0000","last_unstale":"2023-06-08T03:50:29.790036+0000","last_undegraded":"2023-06-08T03:50:29.790036+0000","last_fullsized":"2023-06-08T03:50:29.790036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:59:10.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.692560+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.692035+0000","last_became_peered":"2023-06-08T03:50:24.692035+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790991+0000","last_change":"2023-06-08T03:50:24.698511+0000","last_active":"2023-06-08T03:50:29.790991+0000","last_peered":"2023-06-08T03:50:29.790991+0000","last_clean":"2023-06-08T03:50:29.790991+0000","last_became_active":"2023-06-08T03:50:24.698281+0000","last_became_peered":"2023-06-08T03:50:24.698281+0000","last_unstale":"2023-06-08T03:50:29.790991+0000","last_undegraded":"2023-06-08T03:50:29.790991+0000","last_fullsized":"2023-06-08T03:50:29.790991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:27:46.498562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789941+0000","last_change":"2023-06-08T03:50:24.695969+0000","last_active":"2023-06-08T03:50:29.789941+0000","last_peered":"2023-06-08T03:50:29.789941+0000","last_clean":"2023-06-08T03:50:29.789941+0000","last_became_active":"2023-06-08T03:50:24.695711+0000","last_became_peered":"2023-06-08T03:50:24.695711+0000","last_unstale":"2023-06-08T03:50:29.789941+0000","last_undegraded":"2023-06-08T03:50:29.789941+0000","last_fullsized":"2023-06-08T03:50:29.789941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:45:45.278948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591321+0000","last_change":"2023-06-08T03:50:24.690923+0000","last_active":"2023-06-08T03:50:29.591321+0000","last_peered":"2023-06-08T03:50:29.591321+0000","last_clean":"2023-06-08T03:50:29.591321+0000","last_became_active":"2023-06-08T03:50:24.690667+0000","last_became_peered":"2023-06-08T03:50:24.690667+0000","last_unstale":"2023-06-08T03:50:29.591321+0000","last_undegraded":"2023-06-08T03:50:29.591321+0000","last_fullsized":"2023-06-08T03:50:29.591321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311961+0000","last_change":"2023-06-08T03:50:24.701274+0000","last_active":"2023-06-08T03:50:29.311961+0000","last_peered":"2023-06-08T03:50:29.311961+0000","last_clean":"2023-06-08T03:50:29.311961+0000","last_became_active":"2023-06-08T03:50:24.700991+0000","last_became_peered":"2023-06-08T03:50:24.700991+0000","last_unstale":"2023-06-08T03:50:29.311961+0000","last_undegraded":"2023-06-08T03:50:29.311961+0000","last_fullsized":"2023-06-08T03:50:29.311961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789321+0000","last_change":"2023-06-08T03:50:24.690034+0000","last_active":"2023-06-08T03:50:29.789321+0000","last_peered":"2023-06-08T03:50:29.789321+0000","last_clean":"2023-06-08T03:50:29.789321+0000","last_became_active":"2023-06-08T03:50:24.689524+0000","last_became_peered":"2023-06-08T03:50:24.689524+0000","last_unstale":"2023-06-08T03:50:29.789321+0000","last_undegraded":"2023-06-08T03:50:29.789321+0000","last_fullsized":"2023-06-08T03:50:29.789321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790401+0000","last_change":"2023-06-08T03:50:24.702598+0000","last_active":"2023-06-08T03:50:29.790401+0000","last_peered":"2023-06-08T03:50:29.790401+0000","last_clean":"2023-06-08T03:50:29.790401+0000","last_became_active":"2023-06-08T03:50:24.702337+0000","last_became_peered":"2023-06-08T03:50:24.702337+0000","last_unstale":"2023-06-08T03:50:29.790401+0000","last_undegraded":"2023-06-08T03:50:29.790401+0000","last_fullsized":"2023-06-08T03:50:29.790401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:29:18.553774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789291+0000","last_change":"2023-06-08T03:50:24.689664+0000","last_active":"2023-06-08T03:50:29.789291+0000","last_peered":"2023-06-08T03:50:29.789291+0000","last_clean":"2023-06-08T03:50:29.789291+0000","last_became_active":"2023-06-08T03:50:24.689386+0000","last_became_peered":"2023-06-08T03:50:24.689386+0000","last_unstale":"2023-06-08T03:50:29.789291+0000","last_undegraded":"2023-06-08T03:50:29.789291+0000","last_fullsized":"2023-06-08T03:50:29.789291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:23:34.826399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308547+0000","last_change":"2023-06-08T03:50:24.690551+0000","last_active":"2023-06-08T03:50:29.308547+0000","last_peered":"2023-06-08T03:50:29.308547+0000","last_clean":"2023-06-08T03:50:29.308547+0000","last_became_active":"2023-06-08T03:50:24.689459+0000","last_became_peered":"2023-06-08T03:50:24.689459+0000","last_unstale":"2023-06-08T03:50:29.308547+0000","last_undegraded":"2023-06-08T03:50:29.308547+0000","last_fullsized":"2023-06-08T03:50:29.308547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:21:13.898688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789394+0000","last_change":"2023-06-08T03:50:24.702787+0000","last_active":"2023-06-08T03:50:29.789394+0000","last_peered":"2023-06-08T03:50:29.789394+0000","last_clean":"2023-06-08T03:50:29.789394+0000","last_became_active":"2023-06-08T03:50:24.702390+0000","last_became_peered":"2023-06-08T03:50:24.702390+0000","last_unstale":"2023-06-08T03:50:29.789394+0000","last_undegraded":"2023-06-08T03:50:29.789394+0000","last_fullsized":"2023-06-08T03:50:29.789394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310288+0000","last_change":"2023-06-08T03:50:24.689083+0000","last_active":"2023-06-08T03:50:29.310288+0000","last_peered":"2023-06-08T03:50:29.310288+0000","last_clean":"2023-06-08T03:50:29.310288+0000","last_became_active":"2023-06-08T03:50:24.688803+0000","last_became_peered":"2023-06-08T03:50:24.688803+0000","last_unstale":"2023-06-08T03:50:29.310288+0000","last_undegraded":"2023-06-08T03:50:29.310288+0000","last_fullsized":"2023-06-08T03:50:29.310288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:59:04.642632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789549+0000","last_change":"2023-06-08T03:50:24.689752+0000","last_active":"2023-06-08T03:50:29.789549+0000","last_peered":"2023-06-08T03:50:29.789549+0000","last_clean":"2023-06-08T03:50:29.789549+0000","last_became_active":"2023-06-08T03:50:24.689199+0000","last_became_peered":"2023-06-08T03:50:24.689199+0000","last_unstale":"2023-06-08T03:50:29.789549+0000","last_undegraded":"2023-06-08T03:50:29.789549+0000","last_fullsized":"2023-06-08T03:50:29.789549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:06:49.219275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790465+0000","last_change":"2023-06-08T03:50:24.699485+0000","last_active":"2023-06-08T03:50:29.790465+0000","last_peered":"2023-06-08T03:50:29.790465+0000","last_clean":"2023-06-08T03:50:29.790465+0000","last_became_active":"2023-06-08T03:50:24.699155+0000","last_became_peered":"2023-06-08T03:50:24.699155+0000","last_unstale":"2023-06-08T03:50:29.790465+0000","last_undegraded":"2023-06-08T03:50:29.790465+0000","last_fullsized":"2023-06-08T03:50:29.790465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:09:23.122980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790702+0000","last_change":"2023-06-08T03:50:28.917312+0000","last_active":"2023-06-08T03:50:29.790702+0000","last_peered":"2023-06-08T03:50:29.790702+0000","last_clean":"2023-06-08T03:50:29.790702+0000","last_became_active":"2023-06-08T03:50:28.916901+0000","last_became_peered":"2023-06-08T03:50:28.916901+0000","last_unstale":"2023-06-08T03:50:29.790702+0000","last_undegraded":"2023-06-08T03:50:29.790702+0000","last_fullsized":"2023-06-08T03:50:29.790702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6,0],"acting":[2,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789907+0000","last_change":"2023-06-08T03:50:24.285058+0000","last_active":"2023-06-08T03:50:29.789907+0000","last_peered":"2023-06-08T03:50:29.789907+0000","last_clean":"2023-06-08T03:50:29.789907+0000","last_became_active":"2023-06-08T03:50:24.284770+0000","last_became_peered":"2023-06-08T03:50:24.284770+0000","last_unstale":"2023-06-08T03:50:29.789907+0000","last_undegraded":"2023-06-08T03:50:29.789907+0000","last_fullsized":"2023-06-08T03:50:29.789907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:16:20.267305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789601+0000","last_change":"2023-06-08T03:50:24.698385+0000","last_active":"2023-06-08T03:50:29.789601+0000","last_peered":"2023-06-08T03:50:29.789601+0000","last_clean":"2023-06-08T03:50:29.789601+0000","last_became_active":"2023-06-08T03:50:24.697966+0000","last_became_peered":"2023-06-08T03:50:24.697966+0000","last_unstale":"2023-06-08T03:50:29.789601+0000","last_undegraded":"2023-06-08T03:50:29.789601+0000","last_fullsized":"2023-06-08T03:50:29.789601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:20.947165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791158+0000","last_change":"2023-06-08T03:50:28.922104+0000","last_active":"2023-06-08T03:50:29.791158+0000","last_peered":"2023-06-08T03:50:29.791158+0000","last_clean":"2023-06-08T03:50:29.791158+0000","last_became_active":"2023-06-08T03:50:28.921721+0000","last_became_peered":"2023-06-08T03:50:28.921721+0000","last_unstale":"2023-06-08T03:50:29.791158+0000","last_undegraded":"2023-06-08T03:50:29.791158+0000","last_fullsized":"2023-06-08T03:50:29.791158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:28.031781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,5],"acting":[2,7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308030+0000","last_change":"2023-06-08T03:50:24.286192+0000","last_active":"2023-06-08T03:50:29.308030+0000","last_peered":"2023-06-08T03:50:29.308030+0000","last_clean":"2023-06-08T03:50:29.308030+0000","last_became_active":"2023-06-08T03:50:24.285939+0000","last_became_peered":"2023-06-08T03:50:24.285939+0000","last_unstale":"2023-06-08T03:50:29.308030+0000","last_undegraded":"2023-06-08T03:50:29.308030+0000","last_fullsized":"2023-06-08T03:50:29.308030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:32.185279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308525+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.308525+0000","last_peered":"2023-06-08T03:50:29.308525+0000","last_clean":"2023-06-08T03:50:29.308525+0000","last_became_active":"2023-06-08T03:50:24.688036+0000","last_became_peered":"2023-06-08T03:50:24.688036+0000","last_unstale":"2023-06-08T03:50:29.308525+0000","last_undegraded":"2023-06-08T03:50:29.308525+0000","last_fullsized":"2023-06-08T03:50:29.308525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:23:40.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309722+0000","last_change":"2023-06-08T03:50:28.921813+0000","last_active":"2023-06-08T03:50:29.309722+0000","last_peered":"2023-06-08T03:50:29.309722+0000","last_clean":"2023-06-08T03:50:29.309722+0000","last_became_active":"2023-06-08T03:50:28.921392+0000","last_became_peered":"2023-06-08T03:50:28.921392+0000","last_unstale":"2023-06-08T03:50:29.309722+0000","last_undegraded":"2023-06-08T03:50:29.309722+0000","last_fullsized":"2023-06-08T03:50:29.309722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:18.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2,7],"acting":[6,3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310612+0000","last_change":"2023-06-08T03:50:24.281038+0000","last_active":"2023-06-08T03:50:29.310612+0000","last_peered":"2023-06-08T03:50:29.310612+0000","last_clean":"2023-06-08T03:50:29.310612+0000","last_became_active":"2023-06-08T03:50:24.280771+0000","last_became_peered":"2023-06-08T03:50:24.280771+0000","last_unstale":"2023-06-08T03:50:29.310612+0000","last_undegraded":"2023-06-08T03:50:29.310612+0000","last_fullsized":"2023-06-08T03:50:29.310612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:43.269177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790403+0000","last_change":"2023-06-08T03:50:24.686123+0000","last_active":"2023-06-08T03:50:29.790403+0000","last_peered":"2023-06-08T03:50:29.790403+0000","last_clean":"2023-06-08T03:50:29.790403+0000","last_became_active":"2023-06-08T03:50:24.685951+0000","last_became_peered":"2023-06-08T03:50:24.685951+0000","last_unstale":"2023-06-08T03:50:29.790403+0000","last_undegraded":"2023-06-08T03:50:29.790403+0000","last_fullsized":"2023-06-08T03:50:29.790403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:44:16.167507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789665+0000","last_change":"2023-06-08T03:50:28.929521+0000","last_active":"2023-06-08T03:50:29.789665+0000","last_peered":"2023-06-08T03:50:29.789665+0000","last_clean":"2023-06-08T03:50:29.789665+0000","last_became_active":"2023-06-08T03:50:28.929194+0000","last_became_peered":"2023-06-08T03:50:28.929194+0000","last_unstale":"2023-06-08T03:50:29.789665+0000","last_undegraded":"2023-06-08T03:50:29.789665+0000","last_fullsized":"2023-06-08T03:50:29.789665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:19:17.680617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,5],"acting":[3,2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312228+0000","last_change":"2023-06-08T03:50:24.278075+0000","last_active":"2023-06-08T03:50:29.312228+0000","last_peered":"2023-06-08T03:50:29.312228+0000","last_clean":"2023-06-08T03:50:29.312228+0000","last_became_active":"2023-06-08T03:50:24.277802+0000","last_became_peered":"2023-06-08T03:50:24.277802+0000","last_unstale":"2023-06-08T03:50:29.312228+0000","last_undegraded":"2023-06-08T03:50:29.312228+0000","last_fullsized":"2023-06-08T03:50:29.312228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789217+0000","last_change":"2023-06-08T03:50:24.699630+0000","last_active":"2023-06-08T03:50:29.789217+0000","last_peered":"2023-06-08T03:50:29.789217+0000","last_clean":"2023-06-08T03:50:29.789217+0000","last_became_active":"2023-06-08T03:50:24.699373+0000","last_became_peered":"2023-06-08T03:50:24.699373+0000","last_unstale":"2023-06-08T03:50:29.789217+0000","last_undegraded":"2023-06-08T03:50:29.789217+0000","last_fullsized":"2023-06-08T03:50:29.789217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:08:53.776951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789066+0000","last_change":"2023-06-08T03:50:28.923351+0000","last_active":"2023-06-08T03:50:29.789066+0000","last_peered":"2023-06-08T03:50:29.789066+0000","last_clean":"2023-06-08T03:50:29.789066+0000","last_became_active":"2023-06-08T03:50:28.922947+0000","last_became_peered":"2023-06-08T03:50:28.922947+0000","last_unstale":"2023-06-08T03:50:29.789066+0000","last_undegraded":"2023-06-08T03:50:29.789066+0000","last_fullsized":"2023-06-08T03:50:29.789066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:19:30.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,4],"acting":[2,1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790604+0000","last_change":"2023-06-08T03:50:24.276888+0000","last_active":"2023-06-08T03:50:29.790604+0000","last_peered":"2023-06-08T03:50:29.790604+0000","last_clean":"2023-06-08T03:50:29.790604+0000","last_became_active":"2023-06-08T03:50:24.276608+0000","last_became_peered":"2023-06-08T03:50:24.276608+0000","last_unstale":"2023-06-08T03:50:29.790604+0000","last_undegraded":"2023-06-08T03:50:29.790604+0000","last_fullsized":"2023-06-08T03:50:29.790604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:23.615413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307879+0000","last_change":"2023-06-08T03:50:24.690298+0000","last_active":"2023-06-08T03:50:29.307879+0000","last_peered":"2023-06-08T03:50:29.307879+0000","last_clean":"2023-06-08T03:50:29.307879+0000","last_became_active":"2023-06-08T03:50:24.689874+0000","last_became_peered":"2023-06-08T03:50:24.689874+0000","last_unstale":"2023-06-08T03:50:29.307879+0000","last_undegraded":"2023-06-08T03:50:29.307879+0000","last_fullsized":"2023-06-08T03:50:29.307879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309657+0000","last_change":"2023-06-08T03:50:28.925251+0000","last_active":"2023-06-08T03:50:29.309657+0000","last_peered":"2023-06-08T03:50:29.309657+0000","last_clean":"2023-06-08T03:50:29.309657+0000","last_became_active":"2023-06-08T03:50:28.923741+0000","last_became_peered":"2023-06-08T03:50:28.923741+0000","last_unstale":"2023-06-08T03:50:29.309657+0000","last_undegraded":"2023-06-08T03:50:29.309657+0000","last_fullsized":"2023-06-08T03:50:29.309657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:55:27.532871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7,1],"acting":[4,6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.285463+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.285152+0000","last_became_peered":"2023-06-08T03:50:24.285152+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308197+0000","last_change":"2023-06-08T03:50:24.689181+0000","last_active":"2023-06-08T03:50:29.308197+0000","last_peered":"2023-06-08T03:50:29.308197+0000","last_clean":"2023-06-08T03:50:29.308197+0000","last_became_active":"2023-06-08T03:50:24.688882+0000","last_became_peered":"2023-06-08T03:50:24.688882+0000","last_unstale":"2023-06-08T03:50:29.308197+0000","last_undegraded":"2023-06-08T03:50:29.308197+0000","last_fullsized":"2023-06-08T03:50:29.308197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:58:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308089+0000","last_change":"2023-06-08T03:50:28.919724+0000","last_active":"2023-06-08T03:50:29.308089+0000","last_peered":"2023-06-08T03:50:29.308089+0000","last_clean":"2023-06-08T03:50:29.308089+0000","last_became_active":"2023-06-08T03:50:28.919364+0000","last_became_peered":"2023-06-08T03:50:28.919364+0000","last_unstale":"2023-06-08T03:50:29.308089+0000","last_undegraded":"2023-06-08T03:50:29.308089+0000","last_fullsized":"2023-06-08T03:50:29.308089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:29.657933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4,3],"acting":[6,1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:33.549882+0000","last_change":"2023-06-08T03:50:24.284754+0000","last_active":"2023-06-08T03:50:33.549882+0000","last_peered":"2023-06-08T03:50:33.549882+0000","last_clean":"2023-06-08T03:50:33.549882+0000","last_became_active":"2023-06-08T03:50:24.281521+0000","last_became_peered":"2023-06-08T03:50:24.281521+0000","last_unstale":"2023-06-08T03:50:33.549882+0000","last_undegraded":"2023-06-08T03:50:33.549882+0000","last_fullsized":"2023-06-08T03:50:33.549882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:25.213919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:24.688053+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:24.687402+0000","last_became_peered":"2023-06-08T03:50:24.687402+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:02.669215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790804+0000","last_change":"2023-06-08T03:50:28.918485+0000","last_active":"2023-06-08T03:50:29.790804+0000","last_peered":"2023-06-08T03:50:29.790804+0000","last_clean":"2023-06-08T03:50:29.790804+0000","last_became_active":"2023-06-08T03:50:28.918103+0000","last_became_peered":"2023-06-08T03:50:28.918103+0000","last_unstale":"2023-06-08T03:50:29.790804+0000","last_undegraded":"2023-06-08T03:50:29.790804+0000","last_fullsized":"2023-06-08T03:50:29.790804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:26:43.524009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,6],"acting":[3,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789698+0000","last_change":"2023-06-08T03:50:24.287653+0000","last_active":"2023-06-08T03:50:29.789698+0000","last_peered":"2023-06-08T03:50:29.789698+0000","last_clean":"2023-06-08T03:50:29.789698+0000","last_became_active":"2023-06-08T03:50:24.287374+0000","last_became_peered":"2023-06-08T03:50:24.287374+0000","last_unstale":"2023-06-08T03:50:29.789698+0000","last_undegraded":"2023-06-08T03:50:29.789698+0000","last_fullsized":"2023-06-08T03:50:29.789698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:25:09.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308854+0000","last_change":"2023-06-08T03:50:24.693976+0000","last_active":"2023-06-08T03:50:29.308854+0000","last_peered":"2023-06-08T03:50:29.308854+0000","last_clean":"2023-06-08T03:50:29.308854+0000","last_became_active":"2023-06-08T03:50:24.693510+0000","last_became_peered":"2023-06-08T03:50:24.693510+0000","last_unstale":"2023-06-08T03:50:29.308854+0000","last_undegraded":"2023-06-08T03:50:29.308854+0000","last_fullsized":"2023-06-08T03:50:29.308854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:30.446775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310556+0000","last_change":"2023-06-08T03:50:28.925279+0000","last_active":"2023-06-08T03:50:29.310556+0000","last_peered":"2023-06-08T03:50:29.310556+0000","last_clean":"2023-06-08T03:50:29.310556+0000","last_became_active":"2023-06-08T03:50:28.923608+0000","last_became_peered":"2023-06-08T03:50:28.923608+0000","last_unstale":"2023-06-08T03:50:29.310556+0000","last_undegraded":"2023-06-08T03:50:29.310556+0000","last_fullsized":"2023-06-08T03:50:29.310556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:32.233874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5,7],"acting":[4,2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309210+0000","last_change":"2023-06-08T03:50:24.285047+0000","last_active":"2023-06-08T03:50:29.309210+0000","last_peered":"2023-06-08T03:50:29.309210+0000","last_clean":"2023-06-08T03:50:29.309210+0000","last_became_active":"2023-06-08T03:50:24.284768+0000","last_became_peered":"2023-06-08T03:50:24.284768+0000","last_unstale":"2023-06-08T03:50:29.309210+0000","last_undegraded":"2023-06-08T03:50:29.309210+0000","last_fullsized":"2023-06-08T03:50:29.309210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790899+0000","last_change":"2023-06-08T03:50:24.691967+0000","last_active":"2023-06-08T03:50:29.790899+0000","last_peered":"2023-06-08T03:50:29.790899+0000","last_clean":"2023-06-08T03:50:29.790899+0000","last_became_active":"2023-06-08T03:50:24.691699+0000","last_became_peered":"2023-06-08T03:50:24.691699+0000","last_unstale":"2023-06-08T03:50:29.790899+0000","last_undegraded":"2023-06-08T03:50:29.790899+0000","last_fullsized":"2023-06-08T03:50:29.790899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:40:30.017039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310163+0000","last_change":"2023-06-08T03:50:28.926277+0000","last_active":"2023-06-08T03:50:29.310163+0000","last_peered":"2023-06-08T03:50:29.310163+0000","last_clean":"2023-06-08T03:50:29.310163+0000","last_became_active":"2023-06-08T03:50:28.925921+0000","last_became_peered":"2023-06-08T03:50:28.925921+0000","last_unstale":"2023-06-08T03:50:29.310163+0000","last_undegraded":"2023-06-08T03:50:29.310163+0000","last_fullsized":"2023-06-08T03:50:29.310163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:32:27.673747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1,6],"acting":[4,5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789845+0000","last_change":"2023-06-08T03:50:24.283127+0000","last_active":"2023-06-08T03:50:29.789845+0000","last_peered":"2023-06-08T03:50:29.789845+0000","last_clean":"2023-06-08T03:50:29.789845+0000","last_became_active":"2023-06-08T03:50:24.282515+0000","last_became_peered":"2023-06-08T03:50:24.282515+0000","last_unstale":"2023-06-08T03:50:29.789845+0000","last_undegraded":"2023-06-08T03:50:29.789845+0000","last_fullsized":"2023-06-08T03:50:29.789845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:55.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311940+0000","last_change":"2023-06-08T03:50:24.687906+0000","last_active":"2023-06-08T03:50:29.311940+0000","last_peered":"2023-06-08T03:50:29.311940+0000","last_clean":"2023-06-08T03:50:29.311940+0000","last_became_active":"2023-06-08T03:50:24.687630+0000","last_became_peered":"2023-06-08T03:50:24.687630+0000","last_unstale":"2023-06-08T03:50:29.311940+0000","last_undegraded":"2023-06-08T03:50:29.311940+0000","last_fullsized":"2023-06-08T03:50:29.311940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:12.100669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790030+0000","last_change":"2023-06-08T03:50:28.918543+0000","last_active":"2023-06-08T03:50:29.790030+0000","last_peered":"2023-06-08T03:50:29.790030+0000","last_clean":"2023-06-08T03:50:29.790030+0000","last_became_active":"2023-06-08T03:50:28.918124+0000","last_became_peered":"2023-06-08T03:50:28.918124+0000","last_unstale":"2023-06-08T03:50:29.790030+0000","last_undegraded":"2023-06-08T03:50:29.790030+0000","last_fullsized":"2023-06-08T03:50:29.790030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:32:30.430868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4,0],"acting":[3,6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789271+0000","last_change":"2023-06-08T03:50:24.286716+0000","last_active":"2023-06-08T03:50:29.789271+0000","last_peered":"2023-06-08T03:50:29.789271+0000","last_clean":"2023-06-08T03:50:29.789271+0000","last_became_active":"2023-06-08T03:50:24.286096+0000","last_became_peered":"2023-06-08T03:50:24.286096+0000","last_unstale":"2023-06-08T03:50:29.789271+0000","last_undegraded":"2023-06-08T03:50:29.789271+0000","last_fullsized":"2023-06-08T03:50:29.789271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312273+0000","last_change":"2023-06-08T03:50:24.690535+0000","last_active":"2023-06-08T03:50:29.312273+0000","last_peered":"2023-06-08T03:50:29.312273+0000","last_clean":"2023-06-08T03:50:29.312273+0000","last_became_active":"2023-06-08T03:50:24.689649+0000","last_became_peered":"2023-06-08T03:50:24.689649+0000","last_unstale":"2023-06-08T03:50:29.312273+0000","last_undegraded":"2023-06-08T03:50:29.312273+0000","last_fullsized":"2023-06-08T03:50:29.312273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789697+0000","last_change":"2023-06-08T03:50:28.926759+0000","last_active":"2023-06-08T03:50:29.789697+0000","last_peered":"2023-06-08T03:50:29.789697+0000","last_clean":"2023-06-08T03:50:29.789697+0000","last_became_active":"2023-06-08T03:50:28.926391+0000","last_became_peered":"2023-06-08T03:50:28.926391+0000","last_unstale":"2023-06-08T03:50:29.789697+0000","last_undegraded":"2023-06-08T03:50:29.789697+0000","last_fullsized":"2023-06-08T03:50:29.789697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:28:31.793415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,7],"acting":[1,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.286445+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.286062+0000","last_became_peered":"2023-06-08T03:50:24.286062+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791098+0000","last_change":"2023-06-08T03:50:24.692233+0000","last_active":"2023-06-08T03:50:29.791098+0000","last_peered":"2023-06-08T03:50:29.791098+0000","last_clean":"2023-06-08T03:50:29.791098+0000","last_became_active":"2023-06-08T03:50:24.691584+0000","last_became_peered":"2023-06-08T03:50:24.691584+0000","last_unstale":"2023-06-08T03:50:29.791098+0000","last_undegraded":"2023-06-08T03:50:29.791098+0000","last_fullsized":"2023-06-08T03:50:29.791098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:43.387552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309967+0000","last_change":"2023-06-08T03:50:28.922207+0000","last_active":"2023-06-08T03:50:29.309967+0000","last_peered":"2023-06-08T03:50:29.309967+0000","last_clean":"2023-06-08T03:50:29.309967+0000","last_became_active":"2023-06-08T03:50:28.921809+0000","last_became_peered":"2023-06-08T03:50:28.921809+0000","last_unstale":"2023-06-08T03:50:29.309967+0000","last_undegraded":"2023-06-08T03:50:29.309967+0000","last_fullsized":"2023-06-08T03:50:29.309967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:23:47.506790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,0],"acting":[6,4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790800+0000","last_change":"2023-06-08T03:50:24.280168+0000","last_active":"2023-06-08T03:50:29.790800+0000","last_peered":"2023-06-08T03:50:29.790800+0000","last_clean":"2023-06-08T03:50:29.790800+0000","last_became_active":"2023-06-08T03:50:24.279947+0000","last_became_peered":"2023-06-08T03:50:24.279947+0000","last_unstale":"2023-06-08T03:50:29.790800+0000","last_undegraded":"2023-06-08T03:50:29.790800+0000","last_fullsized":"2023-06-08T03:50:29.790800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:11.091336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790420+0000","last_change":"2023-06-08T03:50:24.702952+0000","last_active":"2023-06-08T03:50:29.790420+0000","last_peered":"2023-06-08T03:50:29.790420+0000","last_clean":"2023-06-08T03:50:29.790420+0000","last_became_active":"2023-06-08T03:50:24.702213+0000","last_became_peered":"2023-06-08T03:50:24.702213+0000","last_unstale":"2023-06-08T03:50:29.790420+0000","last_undegraded":"2023-06-08T03:50:29.790420+0000","last_fullsized":"2023-06-08T03:50:29.790420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:57:59.540147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310770+0000","last_change":"2023-06-08T03:50:28.921768+0000","last_active":"2023-06-08T03:50:29.310770+0000","last_peered":"2023-06-08T03:50:29.310770+0000","last_clean":"2023-06-08T03:50:29.310770+0000","last_became_active":"2023-06-08T03:50:28.921477+0000","last_became_peered":"2023-06-08T03:50:28.921477+0000","last_unstale":"2023-06-08T03:50:29.310770+0000","last_undegraded":"2023-06-08T03:50:29.310770+0000","last_fullsized":"2023-06-08T03:50:29.310770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:06.300830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790697+0000","last_change":"2023-06-08T03:50:24.287775+0000","last_active":"2023-06-08T03:50:29.790697+0000","last_peered":"2023-06-08T03:50:29.790697+0000","last_clean":"2023-06-08T03:50:29.790697+0000","last_became_active":"2023-06-08T03:50:24.287136+0000","last_became_peered":"2023-06-08T03:50:24.287136+0000","last_unstale":"2023-06-08T03:50:29.790697+0000","last_undegraded":"2023-06-08T03:50:29.790697+0000","last_fullsized":"2023-06-08T03:50:29.790697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:17:17.207119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789501+0000","last_change":"2023-06-08T03:50:24.702234+0000","last_active":"2023-06-08T03:50:29.789501+0000","last_peered":"2023-06-08T03:50:29.789501+0000","last_clean":"2023-06-08T03:50:29.789501+0000","last_became_active":"2023-06-08T03:50:24.701881+0000","last_became_peered":"2023-06-08T03:50:24.701881+0000","last_unstale":"2023-06-08T03:50:29.789501+0000","last_undegraded":"2023-06-08T03:50:29.789501+0000","last_fullsized":"2023-06-08T03:50:29.789501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:52:37.709189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308237+0000","last_change":"2023-06-08T03:50:28.914578+0000","last_active":"2023-06-08T03:50:29.308237+0000","last_peered":"2023-06-08T03:50:29.308237+0000","last_clean":"2023-06-08T03:50:29.308237+0000","last_became_active":"2023-06-08T03:50:28.914064+0000","last_became_peered":"2023-06-08T03:50:28.914064+0000","last_unstale":"2023-06-08T03:50:29.308237+0000","last_undegraded":"2023-06-08T03:50:29.308237+0000","last_fullsized":"2023-06-08T03:50:29.308237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:52.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,7],"acting":[6,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.283466+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.283156+0000","last_became_peered":"2023-06-08T03:50:24.283156+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:25.016985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310840+0000","last_change":"2023-06-08T03:50:24.680665+0000","last_active":"2023-06-08T03:50:29.310840+0000","last_peered":"2023-06-08T03:50:29.310840+0000","last_clean":"2023-06-08T03:50:29.310840+0000","last_became_active":"2023-06-08T03:50:24.680424+0000","last_became_peered":"2023-06-08T03:50:24.680424+0000","last_unstale":"2023-06-08T03:50:29.310840+0000","last_undegraded":"2023-06-08T03:50:29.310840+0000","last_fullsized":"2023-06-08T03:50:29.310840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:24.624745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789596+0000","last_change":"2023-06-08T03:50:28.919899+0000","last_active":"2023-06-08T03:50:29.789596+0000","last_peered":"2023-06-08T03:50:29.789596+0000","last_clean":"2023-06-08T03:50:29.789596+0000","last_became_active":"2023-06-08T03:50:28.919515+0000","last_became_peered":"2023-06-08T03:50:28.919515+0000","last_unstale":"2023-06-08T03:50:29.789596+0000","last_undegraded":"2023-06-08T03:50:29.789596+0000","last_fullsized":"2023-06-08T03:50:29.789596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:49:21.771214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5,1],"acting":[7,0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789597+0000","last_change":"2023-06-08T03:50:24.282752+0000","last_active":"2023-06-08T03:50:29.789597+0000","last_peered":"2023-06-08T03:50:29.789597+0000","last_clean":"2023-06-08T03:50:29.789597+0000","last_became_active":"2023-06-08T03:50:24.282473+0000","last_became_peered":"2023-06-08T03:50:24.282473+0000","last_unstale":"2023-06-08T03:50:29.789597+0000","last_undegraded":"2023-06-08T03:50:29.789597+0000","last_fullsized":"2023-06-08T03:50:29.789597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:15:57.351189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591060+0000","last_change":"2023-06-08T03:50:24.691781+0000","last_active":"2023-06-08T03:50:29.591060+0000","last_peered":"2023-06-08T03:50:29.591060+0000","last_clean":"2023-06-08T03:50:29.591060+0000","last_became_active":"2023-06-08T03:50:24.690055+0000","last_became_peered":"2023-06-08T03:50:24.690055+0000","last_unstale":"2023-06-08T03:50:29.591060+0000","last_undegraded":"2023-06-08T03:50:29.591060+0000","last_fullsized":"2023-06-08T03:50:29.591060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311854+0000","last_change":"2023-06-08T03:50:28.927198+0000","last_active":"2023-06-08T03:50:29.311854+0000","last_peered":"2023-06-08T03:50:29.311854+0000","last_clean":"2023-06-08T03:50:29.311854+0000","last_became_active":"2023-06-08T03:50:28.926812+0000","last_became_peered":"2023-06-08T03:50:28.926812+0000","last_unstale":"2023-06-08T03:50:29.311854+0000","last_undegraded":"2023-06-08T03:50:29.311854+0000","last_fullsized":"2023-06-08T03:50:29.311854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4,2],"acting":[5,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789506+0000","last_change":"2023-06-08T03:50:24.287767+0000","last_active":"2023-06-08T03:50:29.789506+0000","last_peered":"2023-06-08T03:50:29.789506+0000","last_clean":"2023-06-08T03:50:29.789506+0000","last_became_active":"2023-06-08T03:50:24.287060+0000","last_became_peered":"2023-06-08T03:50:24.287060+0000","last_unstale":"2023-06-08T03:50:29.789506+0000","last_undegraded":"2023-06-08T03:50:29.789506+0000","last_fullsized":"2023-06-08T03:50:29.789506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:27.592926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790578+0000","last_change":"2023-06-08T03:50:24.700545+0000","last_active":"2023-06-08T03:50:29.790578+0000","last_peered":"2023-06-08T03:50:29.790578+0000","last_clean":"2023-06-08T03:50:29.790578+0000","last_became_active":"2023-06-08T03:50:24.700276+0000","last_became_peered":"2023-06-08T03:50:24.700276+0000","last_unstale":"2023-06-08T03:50:29.790578+0000","last_undegraded":"2023-06-08T03:50:29.790578+0000","last_fullsized":"2023-06-08T03:50:29.790578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:50.252975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791082+0000","last_change":"2023-06-08T03:50:28.920817+0000","last_active":"2023-06-08T03:50:29.791082+0000","last_peered":"2023-06-08T03:50:29.791082+0000","last_clean":"2023-06-08T03:50:29.791082+0000","last_became_active":"2023-06-08T03:50:28.920485+0000","last_became_peered":"2023-06-08T03:50:28.920485+0000","last_unstale":"2023-06-08T03:50:29.791082+0000","last_undegraded":"2023-06-08T03:50:29.791082+0000","last_fullsized":"2023-06-08T03:50:29.791082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:36.826227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,5],"acting":[1,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790009+0000","last_change":"2023-06-08T03:50:24.283331+0000","last_active":"2023-06-08T03:50:29.790009+0000","last_peered":"2023-06-08T03:50:29.790009+0000","last_clean":"2023-06-08T03:50:29.790009+0000","last_became_active":"2023-06-08T03:50:24.282856+0000","last_became_peered":"2023-06-08T03:50:24.282856+0000","last_unstale":"2023-06-08T03:50:29.790009+0000","last_undegraded":"2023-06-08T03:50:29.790009+0000","last_fullsized":"2023-06-08T03:50:29.790009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:52:45.337874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790917+0000","last_change":"2023-06-08T03:50:24.683795+0000","last_active":"2023-06-08T03:50:29.790917+0000","last_peered":"2023-06-08T03:50:29.790917+0000","last_clean":"2023-06-08T03:50:29.790917+0000","last_became_active":"2023-06-08T03:50:24.683510+0000","last_became_peered":"2023-06-08T03:50:24.683510+0000","last_unstale":"2023-06-08T03:50:29.790917+0000","last_undegraded":"2023-06-08T03:50:29.790917+0000","last_fullsized":"2023-06-08T03:50:29.790917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790162+0000","last_change":"2023-06-08T03:50:28.920202+0000","last_active":"2023-06-08T03:50:29.790162+0000","last_peered":"2023-06-08T03:50:29.790162+0000","last_clean":"2023-06-08T03:50:29.790162+0000","last_became_active":"2023-06-08T03:50:28.919960+0000","last_became_peered":"2023-06-08T03:50:28.919960+0000","last_unstale":"2023-06-08T03:50:29.790162+0000","last_undegraded":"2023-06-08T03:50:29.790162+0000","last_fullsized":"2023-06-08T03:50:29.790162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:44.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4,5],"acting":[3,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307873+0000","last_change":"2023-06-08T03:50:24.284333+0000","last_active":"2023-06-08T03:50:29.307873+0000","last_peered":"2023-06-08T03:50:29.307873+0000","last_clean":"2023-06-08T03:50:29.307873+0000","last_became_active":"2023-06-08T03:50:24.284039+0000","last_became_peered":"2023-06-08T03:50:24.284039+0000","last_unstale":"2023-06-08T03:50:29.307873+0000","last_undegraded":"2023-06-08T03:50:29.307873+0000","last_fullsized":"2023-06-08T03:50:29.307873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:37:13.194212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789927+0000","last_change":"2023-06-08T03:50:24.691444+0000","last_active":"2023-06-08T03:50:29.789927+0000","last_peered":"2023-06-08T03:50:29.789927+0000","last_clean":"2023-06-08T03:50:29.789927+0000","last_became_active":"2023-06-08T03:50:24.690971+0000","last_became_peered":"2023-06-08T03:50:24.690971+0000","last_unstale":"2023-06-08T03:50:29.789927+0000","last_undegraded":"2023-06-08T03:50:29.789927+0000","last_fullsized":"2023-06-08T03:50:29.789927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:38:23.297572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790270+0000","last_change":"2023-06-08T03:50:28.929890+0000","last_active":"2023-06-08T03:50:29.790270+0000","last_peered":"2023-06-08T03:50:29.790270+0000","last_clean":"2023-06-08T03:50:29.790270+0000","last_became_active":"2023-06-08T03:50:28.929496+0000","last_became_peered":"2023-06-08T03:50:28.929496+0000","last_unstale":"2023-06-08T03:50:29.790270+0000","last_undegraded":"2023-06-08T03:50:29.790270+0000","last_fullsized":"2023-06-08T03:50:29.790270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:21:51.680343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310379+0000","last_change":"2023-06-08T03:50:24.282724+0000","last_active":"2023-06-08T03:50:29.310379+0000","last_peered":"2023-06-08T03:50:29.310379+0000","last_clean":"2023-06-08T03:50:29.310379+0000","last_became_active":"2023-06-08T03:50:24.282451+0000","last_became_peered":"2023-06-08T03:50:24.282451+0000","last_unstale":"2023-06-08T03:50:29.310379+0000","last_undegraded":"2023-06-08T03:50:29.310379+0000","last_fullsized":"2023-06-08T03:50:29.310379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:44:39.775596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790625+0000","last_change":"2023-06-08T03:50:19.804887+0000","last_active":"2023-06-08T03:50:29.790625+0000","last_peered":"2023-06-08T03:50:29.790625+0000","last_clean":"2023-06-08T03:50:29.790625+0000","last_became_active":"2023-06-08T03:50:19.804546+0000","last_became_peered":"2023-06-08T03:50:19.804546+0000","last_unstale":"2023-06-08T03:50:29.790625+0000","last_undegraded":"2023-06-08T03:50:29.790625+0000","last_fullsized":"2023-06-08T03:50:29.790625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:34.859632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307694+0000","last_change":"2023-06-08T03:50:24.686843+0000","last_active":"2023-06-08T03:50:29.307694+0000","last_peered":"2023-06-08T03:50:29.307694+0000","last_clean":"2023-06-08T03:50:29.307694+0000","last_became_active":"2023-06-08T03:50:24.686587+0000","last_became_peered":"2023-06-08T03:50:24.686587+0000","last_unstale":"2023-06-08T03:50:29.307694+0000","last_undegraded":"2023-06-08T03:50:29.307694+0000","last_fullsized":"2023-06-08T03:50:29.307694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:36.573402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308557+0000","last_change":"2023-06-08T03:50:28.924976+0000","last_active":"2023-06-08T03:50:29.308557+0000","last_peered":"2023-06-08T03:50:29.308557+0000","last_clean":"2023-06-08T03:50:29.308557+0000","last_became_active":"2023-06-08T03:50:28.924335+0000","last_became_peered":"2023-06-08T03:50:28.924335+0000","last_unstale":"2023-06-08T03:50:29.308557+0000","last_undegraded":"2023-06-08T03:50:29.308557+0000","last_fullsized":"2023-06-08T03:50:29.308557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:47:44.099384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789646+0000","last_change":"2023-06-08T03:50:19.172271+0000","last_active":"2023-06-08T03:50:29.789646+0000","last_peered":"2023-06-08T03:50:29.789646+0000","last_clean":"2023-06-08T03:50:29.789646+0000","last_became_active":"2023-06-08T03:50:19.171917+0000","last_became_peered":"2023-06-08T03:50:19.171917+0000","last_unstale":"2023-06-08T03:50:29.789646+0000","last_undegraded":"2023-06-08T03:50:29.789646+0000","last_fullsized":"2023-06-08T03:50:29.789646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:01:56.709362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308387+0000","last_change":"2023-06-08T03:50:24.284253+0000","last_active":"2023-06-08T03:50:29.308387+0000","last_peered":"2023-06-08T03:50:29.308387+0000","last_clean":"2023-06-08T03:50:29.308387+0000","last_became_active":"2023-06-08T03:50:24.283958+0000","last_became_peered":"2023-06-08T03:50:24.283958+0000","last_unstale":"2023-06-08T03:50:29.308387+0000","last_undegraded":"2023-06-08T03:50:29.308387+0000","last_fullsized":"2023-06-08T03:50:29.308387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:02:34.479410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.690498+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.690242+0000","last_became_peered":"2023-06-08T03:50:24.690242+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:23:08.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790091+0000","last_change":"2023-06-08T03:50:28.921864+0000","last_active":"2023-06-08T03:50:29.790091+0000","last_peered":"2023-06-08T03:50:29.790091+0000","last_clean":"2023-06-08T03:50:29.790091+0000","last_became_active":"2023-06-08T03:50:28.921426+0000","last_became_peered":"2023-06-08T03:50:28.921426+0000","last_unstale":"2023-06-08T03:50:29.790091+0000","last_undegraded":"2023-06-08T03:50:29.790091+0000","last_fullsized":"2023-06-08T03:50:29.790091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:43:24.342082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,0],"acting":[3,7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309435+0000","last_change":"2023-06-08T03:50:19.171073+0000","last_active":"2023-06-08T03:50:29.309435+0000","last_peered":"2023-06-08T03:50:29.309435+0000","last_clean":"2023-06-08T03:50:29.309435+0000","last_became_active":"2023-06-08T03:50:19.170775+0000","last_became_peered":"2023-06-08T03:50:19.170775+0000","last_unstale":"2023-06-08T03:50:29.309435+0000","last_undegraded":"2023-06-08T03:50:29.309435+0000","last_fullsized":"2023-06-08T03:50:29.309435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:51:24.758705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791212+0000","last_change":"2023-06-08T03:50:19.167800+0000","last_active":"2023-06-08T03:50:29.791212+0000","last_peered":"2023-06-08T03:50:29.791212+0000","last_clean":"2023-06-08T03:50:29.791212+0000","last_became_active":"2023-06-08T03:50:19.167412+0000","last_became_peered":"2023-06-08T03:50:19.167412+0000","last_unstale":"2023-06-08T03:50:29.791212+0000","last_undegraded":"2023-06-08T03:50:29.791212+0000","last_fullsized":"2023-06-08T03:50:29.791212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:45:23.552252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309769+0000","last_change":"2023-06-08T03:50:24.280635+0000","last_active":"2023-06-08T03:50:29.309769+0000","last_peered":"2023-06-08T03:50:29.309769+0000","last_clean":"2023-06-08T03:50:29.309769+0000","last_became_active":"2023-06-08T03:50:24.280376+0000","last_became_peered":"2023-06-08T03:50:24.280376+0000","last_unstale":"2023-06-08T03:50:29.309769+0000","last_undegraded":"2023-06-08T03:50:29.309769+0000","last_fullsized":"2023-06-08T03:50:29.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:41.269585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312281+0000","last_change":"2023-06-08T03:50:24.685266+0000","last_active":"2023-06-08T03:50:29.312281+0000","last_peered":"2023-06-08T03:50:29.312281+0000","last_clean":"2023-06-08T03:50:29.312281+0000","last_became_active":"2023-06-08T03:50:24.684970+0000","last_became_peered":"2023-06-08T03:50:24.684970+0000","last_unstale":"2023-06-08T03:50:29.312281+0000","last_undegraded":"2023-06-08T03:50:29.312281+0000","last_fullsized":"2023-06-08T03:50:29.312281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789896+0000","last_change":"2023-06-08T03:50:28.926537+0000","last_active":"2023-06-08T03:50:29.789896+0000","last_peered":"2023-06-08T03:50:29.789896+0000","last_clean":"2023-06-08T03:50:29.789896+0000","last_became_active":"2023-06-08T03:50:28.926095+0000","last_became_peered":"2023-06-08T03:50:28.926095+0000","last_unstale":"2023-06-08T03:50:29.789896+0000","last_undegraded":"2023-06-08T03:50:29.789896+0000","last_fullsized":"2023-06-08T03:50:29.789896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:16:59.025169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,1],"acting":[7,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590877+0000","last_change":"2023-06-08T03:50:19.171834+0000","last_active":"2023-06-08T03:50:29.590877+0000","last_peered":"2023-06-08T03:50:29.590877+0000","last_clean":"2023-06-08T03:50:29.590877+0000","last_became_active":"2023-06-08T03:50:19.171541+0000","last_became_peered":"2023-06-08T03:50:19.171541+0000","last_unstale":"2023-06-08T03:50:29.590877+0000","last_undegraded":"2023-06-08T03:50:29.590877+0000","last_fullsized":"2023-06-08T03:50:29.590877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:42.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310150+0000","last_change":"2023-06-08T03:50:24.275533+0000","last_active":"2023-06-08T03:50:29.310150+0000","last_peered":"2023-06-08T03:50:29.310150+0000","last_clean":"2023-06-08T03:50:29.310150+0000","last_became_active":"2023-06-08T03:50:24.275086+0000","last_became_peered":"2023-06-08T03:50:24.275086+0000","last_unstale":"2023-06-08T03:50:29.310150+0000","last_undegraded":"2023-06-08T03:50:29.310150+0000","last_fullsized":"2023-06-08T03:50:29.310150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:49.514141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309093+0000","last_change":"2023-06-08T03:50:24.694692+0000","last_active":"2023-06-08T03:50:29.309093+0000","last_peered":"2023-06-08T03:50:29.309093+0000","last_clean":"2023-06-08T03:50:29.309093+0000","last_became_active":"2023-06-08T03:50:24.694404+0000","last_became_peered":"2023-06-08T03:50:24.694404+0000","last_unstale":"2023-06-08T03:50:29.309093+0000","last_undegraded":"2023-06-08T03:50:29.309093+0000","last_fullsized":"2023-06-08T03:50:29.309093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:06.873559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591398+0000","last_change":"2023-06-08T03:50:28.926999+0000","last_active":"2023-06-08T03:50:29.591398+0000","last_peered":"2023-06-08T03:50:29.591398+0000","last_clean":"2023-06-08T03:50:29.591398+0000","last_became_active":"2023-06-08T03:50:28.926614+0000","last_became_peered":"2023-06-08T03:50:28.926614+0000","last_unstale":"2023-06-08T03:50:29.591398+0000","last_undegraded":"2023-06-08T03:50:29.591398+0000","last_fullsized":"2023-06-08T03:50:29.591398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:45.910533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789240+0000","last_change":"2023-06-08T03:50:19.170601+0000","last_active":"2023-06-08T03:50:29.789240+0000","last_peered":"2023-06-08T03:50:29.789240+0000","last_clean":"2023-06-08T03:50:29.789240+0000","last_became_active":"2023-06-08T03:50:19.170344+0000","last_became_peered":"2023-06-08T03:50:19.170344+0000","last_unstale":"2023-06-08T03:50:29.789240+0000","last_undegraded":"2023-06-08T03:50:29.789240+0000","last_fullsized":"2023-06-08T03:50:29.789240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:35.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307871+0000","last_change":"2023-06-08T03:50:24.280451+0000","last_active":"2023-06-08T03:50:29.307871+0000","last_peered":"2023-06-08T03:50:29.307871+0000","last_clean":"2023-06-08T03:50:29.307871+0000","last_became_active":"2023-06-08T03:50:24.280118+0000","last_became_peered":"2023-06-08T03:50:24.280118+0000","last_unstale":"2023-06-08T03:50:29.307871+0000","last_undegraded":"2023-06-08T03:50:29.307871+0000","last_fullsized":"2023-06-08T03:50:29.307871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:18.146699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790813+0000","last_change":"2023-06-08T03:50:24.692660+0000","last_active":"2023-06-08T03:50:29.790813+0000","last_peered":"2023-06-08T03:50:29.790813+0000","last_clean":"2023-06-08T03:50:29.790813+0000","last_became_active":"2023-06-08T03:50:24.692373+0000","last_became_peered":"2023-06-08T03:50:24.692373+0000","last_unstale":"2023-06-08T03:50:29.790813+0000","last_undegraded":"2023-06-08T03:50:29.790813+0000","last_fullsized":"2023-06-08T03:50:29.790813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:14:48.949961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789804+0000","last_change":"2023-06-08T03:50:28.924634+0000","last_active":"2023-06-08T03:50:29.789804+0000","last_peered":"2023-06-08T03:50:29.789804+0000","last_clean":"2023-06-08T03:50:29.789804+0000","last_became_active":"2023-06-08T03:50:28.924227+0000","last_became_peered":"2023-06-08T03:50:28.924227+0000","last_unstale":"2023-06-08T03:50:29.789804+0000","last_undegraded":"2023-06-08T03:50:29.789804+0000","last_fullsized":"2023-06-08T03:50:29.789804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:24:05.389046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3,1],"acting":[7,4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789742+0000","last_change":"2023-06-08T03:50:19.805195+0000","last_active":"2023-06-08T03:50:29.789742+0000","last_peered":"2023-06-08T03:50:29.789742+0000","last_clean":"2023-06-08T03:50:29.789742+0000","last_became_active":"2023-06-08T03:50:19.804708+0000","last_became_peered":"2023-06-08T03:50:19.804708+0000","last_unstale":"2023-06-08T03:50:29.789742+0000","last_undegraded":"2023-06-08T03:50:29.789742+0000","last_fullsized":"2023-06-08T03:50:29.789742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:53:26.398016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591012+0000","last_change":"2023-06-08T03:50:24.281698+0000","last_active":"2023-06-08T03:50:29.591012+0000","last_peered":"2023-06-08T03:50:29.591012+0000","last_clean":"2023-06-08T03:50:29.591012+0000","last_became_active":"2023-06-08T03:50:24.281443+0000","last_became_peered":"2023-06-08T03:50:24.281443+0000","last_unstale":"2023-06-08T03:50:29.591012+0000","last_undegraded":"2023-06-08T03:50:29.591012+0000","last_fullsized":"2023-06-08T03:50:29.591012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:29.437443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309408+0000","last_change":"2023-06-08T03:50:24.693777+0000","last_active":"2023-06-08T03:50:29.309408+0000","last_peered":"2023-06-08T03:50:29.309408+0000","last_clean":"2023-06-08T03:50:29.309408+0000","last_became_active":"2023-06-08T03:50:24.693193+0000","last_became_peered":"2023-06-08T03:50:24.693193+0000","last_unstale":"2023-06-08T03:50:29.309408+0000","last_undegraded":"2023-06-08T03:50:29.309408+0000","last_fullsized":"2023-06-08T03:50:29.309408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591049+0000","last_change":"2023-06-08T03:50:28.926459+0000","last_active":"2023-06-08T03:50:29.591049+0000","last_peered":"2023-06-08T03:50:29.591049+0000","last_clean":"2023-06-08T03:50:29.591049+0000","last_became_active":"2023-06-08T03:50:28.926041+0000","last_became_peered":"2023-06-08T03:50:28.926041+0000","last_unstale":"2023-06-08T03:50:29.591049+0000","last_undegraded":"2023-06-08T03:50:29.591049+0000","last_fullsized":"2023-06-08T03:50:29.591049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:10.505175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,5],"acting":[0,7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:19.805012+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:19.804722+0000","last_became_peered":"2023-06-08T03:50:19.804722+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:43:32.916770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309261+0000","last_change":"2023-06-08T03:50:24.284390+0000","last_active":"2023-06-08T03:50:29.309261+0000","last_peered":"2023-06-08T03:50:29.309261+0000","last_clean":"2023-06-08T03:50:29.309261+0000","last_became_active":"2023-06-08T03:50:24.284129+0000","last_became_peered":"2023-06-08T03:50:24.284129+0000","last_unstale":"2023-06-08T03:50:29.309261+0000","last_undegraded":"2023-06-08T03:50:29.309261+0000","last_fullsized":"2023-06-08T03:50:29.309261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309328+0000","last_change":"2023-06-08T03:50:24.693789+0000","last_active":"2023-06-08T03:50:29.309328+0000","last_peered":"2023-06-08T03:50:29.309328+0000","last_clean":"2023-06-08T03:50:29.309328+0000","last_became_active":"2023-06-08T03:50:24.693205+0000","last_became_peered":"2023-06-08T03:50:24.693205+0000","last_unstale":"2023-06-08T03:50:29.309328+0000","last_undegraded":"2023-06-08T03:50:29.309328+0000","last_fullsized":"2023-06-08T03:50:29.309328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310499+0000","last_change":"2023-06-08T03:50:28.924218+0000","last_active":"2023-06-08T03:50:29.310499+0000","last_peered":"2023-06-08T03:50:29.310499+0000","last_clean":"2023-06-08T03:50:29.310499+0000","last_became_active":"2023-06-08T03:50:28.923473+0000","last_became_peered":"2023-06-08T03:50:28.923473+0000","last_unstale":"2023-06-08T03:50:29.310499+0000","last_undegraded":"2023-06-08T03:50:29.310499+0000","last_fullsized":"2023-06-08T03:50:29.310499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:11:10.428932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3,1],"acting":[4,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790922+0000","last_change":"2023-06-08T03:50:19.805107+0000","last_active":"2023-06-08T03:50:29.790922+0000","last_peered":"2023-06-08T03:50:29.790922+0000","last_clean":"2023-06-08T03:50:29.790922+0000","last_became_active":"2023-06-08T03:50:19.804754+0000","last_became_peered":"2023-06-08T03:50:19.804754+0000","last_unstale":"2023-06-08T03:50:29.790922+0000","last_undegraded":"2023-06-08T03:50:29.790922+0000","last_fullsized":"2023-06-08T03:50:29.790922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:13.049279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790682+0000","last_change":"2023-06-08T03:50:24.281311+0000","last_active":"2023-06-08T03:50:29.790682+0000","last_peered":"2023-06-08T03:50:29.790682+0000","last_clean":"2023-06-08T03:50:29.790682+0000","last_became_active":"2023-06-08T03:50:24.280858+0000","last_became_peered":"2023-06-08T03:50:24.280858+0000","last_unstale":"2023-06-08T03:50:29.790682+0000","last_undegraded":"2023-06-08T03:50:29.790682+0000","last_fullsized":"2023-06-08T03:50:29.790682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:01.113573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309301+0000","last_change":"2023-06-08T03:50:24.700010+0000","last_active":"2023-06-08T03:50:29.309301+0000","last_peered":"2023-06-08T03:50:29.309301+0000","last_clean":"2023-06-08T03:50:29.309301+0000","last_became_active":"2023-06-08T03:50:24.699801+0000","last_became_peered":"2023-06-08T03:50:24.699801+0000","last_unstale":"2023-06-08T03:50:29.309301+0000","last_undegraded":"2023-06-08T03:50:29.309301+0000","last_fullsized":"2023-06-08T03:50:29.309301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308391+0000","last_change":"2023-06-08T03:50:28.920073+0000","last_active":"2023-06-08T03:50:29.308391+0000","last_peered":"2023-06-08T03:50:29.308391+0000","last_clean":"2023-06-08T03:50:29.308391+0000","last_became_active":"2023-06-08T03:50:28.919601+0000","last_became_peered":"2023-06-08T03:50:28.919601+0000","last_unstale":"2023-06-08T03:50:29.308391+0000","last_undegraded":"2023-06-08T03:50:29.308391+0000","last_fullsized":"2023-06-08T03:50:29.308391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:38:02.965137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309419+0000","last_change":"2023-06-08T03:50:24.285734+0000","last_active":"2023-06-08T03:50:29.309419+0000","last_peered":"2023-06-08T03:50:29.309419+0000","last_clean":"2023-06-08T03:50:29.309419+0000","last_became_active":"2023-06-08T03:50:24.285421+0000","last_became_peered":"2023-06-08T03:50:24.285421+0000","last_unstale":"2023-06-08T03:50:29.309419+0000","last_undegraded":"2023-06-08T03:50:29.309419+0000","last_fullsized":"2023-06-08T03:50:29.309419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:13:35.050524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309586+0000","last_change":"2023-06-08T03:50:24.696432+0000","last_active":"2023-06-08T03:50:29.309586+0000","last_peered":"2023-06-08T03:50:29.309586+0000","last_clean":"2023-06-08T03:50:29.309586+0000","last_became_active":"2023-06-08T03:50:24.695650+0000","last_became_peered":"2023-06-08T03:50:24.695650+0000","last_unstale":"2023-06-08T03:50:29.309586+0000","last_undegraded":"2023-06-08T03:50:29.309586+0000","last_fullsized":"2023-06-08T03:50:29.309586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:33:18.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790496+0000","last_change":"2023-06-08T03:50:28.929940+0000","last_active":"2023-06-08T03:50:29.790496+0000","last_peered":"2023-06-08T03:50:29.790496+0000","last_clean":"2023-06-08T03:50:29.790496+0000","last_became_active":"2023-06-08T03:50:28.929641+0000","last_became_peered":"2023-06-08T03:50:28.929641+0000","last_unstale":"2023-06-08T03:50:29.790496+0000","last_undegraded":"2023-06-08T03:50:29.790496+0000","last_fullsized":"2023-06-08T03:50:29.790496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:40.205018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.285199+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.284917+0000","last_became_peered":"2023-06-08T03:50:24.284917+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790180+0000","last_change":"2023-06-08T03:50:24.680708+0000","last_active":"2023-06-08T03:50:29.790180+0000","last_peered":"2023-06-08T03:50:29.790180+0000","last_clean":"2023-06-08T03:50:29.790180+0000","last_became_active":"2023-06-08T03:50:24.680437+0000","last_became_peered":"2023-06-08T03:50:24.680437+0000","last_unstale":"2023-06-08T03:50:29.790180+0000","last_undegraded":"2023-06-08T03:50:29.790180+0000","last_fullsized":"2023-06-08T03:50:29.790180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:44.278509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789986+0000","last_change":"2023-06-08T03:50:28.922303+0000","last_active":"2023-06-08T03:50:29.789986+0000","last_peered":"2023-06-08T03:50:29.789986+0000","last_clean":"2023-06-08T03:50:29.789986+0000","last_became_active":"2023-06-08T03:50:28.921661+0000","last_became_peered":"2023-06-08T03:50:28.921661+0000","last_unstale":"2023-06-08T03:50:29.789986+0000","last_undegraded":"2023-06-08T03:50:29.789986+0000","last_fullsized":"2023-06-08T03:50:29.789986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:54.155089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,4],"acting":[2,7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790594+0000","last_change":"2023-06-08T03:50:24.287963+0000","last_active":"2023-06-08T03:50:29.790594+0000","last_peered":"2023-06-08T03:50:29.790594+0000","last_clean":"2023-06-08T03:50:29.790594+0000","last_became_active":"2023-06-08T03:50:24.287513+0000","last_became_peered":"2023-06-08T03:50:24.287513+0000","last_unstale":"2023-06-08T03:50:29.790594+0000","last_undegraded":"2023-06-08T03:50:29.790594+0000","last_fullsized":"2023-06-08T03:50:29.790594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:41:31.366674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789388+0000","last_change":"2023-06-08T03:50:24.694656+0000","last_active":"2023-06-08T03:50:29.789388+0000","last_peered":"2023-06-08T03:50:29.789388+0000","last_clean":"2023-06-08T03:50:29.789388+0000","last_became_active":"2023-06-08T03:50:24.694387+0000","last_became_peered":"2023-06-08T03:50:24.694387+0000","last_unstale":"2023-06-08T03:50:29.789388+0000","last_undegraded":"2023-06-08T03:50:29.789388+0000","last_fullsized":"2023-06-08T03:50:29.789388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:11:50.398160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789148+0000","last_change":"2023-06-08T03:50:28.926194+0000","last_active":"2023-06-08T03:50:29.789148+0000","last_peered":"2023-06-08T03:50:29.789148+0000","last_clean":"2023-06-08T03:50:29.789148+0000","last_became_active":"2023-06-08T03:50:28.925783+0000","last_became_peered":"2023-06-08T03:50:28.925783+0000","last_unstale":"2023-06-08T03:50:29.789148+0000","last_undegraded":"2023-06-08T03:50:29.789148+0000","last_fullsized":"2023-06-08T03:50:29.789148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:26.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5,7],"acting":[1,0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312099+0000","last_change":"2023-06-08T03:50:24.277779+0000","last_active":"2023-06-08T03:50:29.312099+0000","last_peered":"2023-06-08T03:50:29.312099+0000","last_clean":"2023-06-08T03:50:29.312099+0000","last_became_active":"2023-06-08T03:50:24.277493+0000","last_became_peered":"2023-06-08T03:50:24.277493+0000","last_unstale":"2023-06-08T03:50:29.312099+0000","last_undegraded":"2023-06-08T03:50:29.312099+0000","last_fullsized":"2023-06-08T03:50:29.312099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789292+0000","last_change":"2023-06-08T03:50:24.696150+0000","last_active":"2023-06-08T03:50:29.789292+0000","last_peered":"2023-06-08T03:50:29.789292+0000","last_clean":"2023-06-08T03:50:29.789292+0000","last_became_active":"2023-06-08T03:50:24.695605+0000","last_became_peered":"2023-06-08T03:50:24.695605+0000","last_unstale":"2023-06-08T03:50:29.789292+0000","last_undegraded":"2023-06-08T03:50:29.789292+0000","last_fullsized":"2023-06-08T03:50:29.789292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:28:06.091963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308122+0000","last_change":"2023-06-08T03:50:28.924072+0000","last_active":"2023-06-08T03:50:29.308122+0000","last_peered":"2023-06-08T03:50:29.308122+0000","last_clean":"2023-06-08T03:50:29.308122+0000","last_became_active":"2023-06-08T03:50:28.923155+0000","last_became_peered":"2023-06-08T03:50:28.923155+0000","last_unstale":"2023-06-08T03:50:29.308122+0000","last_undegraded":"2023-06-08T03:50:29.308122+0000","last_fullsized":"2023-06-08T03:50:29.308122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:00:25.670438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1,0],"acting":[4,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312115+0000","last_change":"2023-06-08T03:50:24.285764+0000","last_active":"2023-06-08T03:50:29.312115+0000","last_peered":"2023-06-08T03:50:29.312115+0000","last_clean":"2023-06-08T03:50:29.312115+0000","last_became_active":"2023-06-08T03:50:24.285477+0000","last_became_peered":"2023-06-08T03:50:24.285477+0000","last_unstale":"2023-06-08T03:50:29.312115+0000","last_undegraded":"2023-06-08T03:50:29.312115+0000","last_fullsized":"2023-06-08T03:50:29.312115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:41.022058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.788785+0000","last_change":"2023-06-08T03:50:24.679506+0000","last_active":"2023-06-08T03:50:29.788785+0000","last_peered":"2023-06-08T03:50:29.788785+0000","last_clean":"2023-06-08T03:50:29.788785+0000","last_became_active":"2023-06-08T03:50:24.679246+0000","last_became_peered":"2023-06-08T03:50:24.679246+0000","last_unstale":"2023-06-08T03:50:29.788785+0000","last_undegraded":"2023-06-08T03:50:29.788785+0000","last_fullsized":"2023-06-08T03:50:29.788785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:53:20.880498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789376+0000","last_change":"2023-06-08T03:50:24.690393+0000","last_active":"2023-06-08T03:50:29.789376+0000","last_peered":"2023-06-08T03:50:29.789376+0000","last_clean":"2023-06-08T03:50:29.789376+0000","last_became_active":"2023-06-08T03:50:24.690144+0000","last_became_peered":"2023-06-08T03:50:24.690144+0000","last_unstale":"2023-06-08T03:50:29.789376+0000","last_undegraded":"2023-06-08T03:50:29.789376+0000","last_fullsized":"2023-06-08T03:50:29.789376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309500+0000","last_change":"2023-06-08T03:50:24.693278+0000","last_active":"2023-06-08T03:50:29.309500+0000","last_peered":"2023-06-08T03:50:29.309500+0000","last_clean":"2023-06-08T03:50:29.309500+0000","last_became_active":"2023-06-08T03:50:24.692978+0000","last_became_peered":"2023-06-08T03:50:24.692978+0000","last_unstale":"2023-06-08T03:50:29.309500+0000","last_undegraded":"2023-06-08T03:50:29.309500+0000","last_fullsized":"2023-06-08T03:50:29.309500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591028+0000","last_change":"2023-06-08T03:50:24.700428+0000","last_active":"2023-06-08T03:50:29.591028+0000","last_peered":"2023-06-08T03:50:29.591028+0000","last_clean":"2023-06-08T03:50:29.591028+0000","last_became_active":"2023-06-08T03:50:24.700158+0000","last_became_peered":"2023-06-08T03:50:24.700158+0000","last_unstale":"2023-06-08T03:50:29.591028+0000","last_undegraded":"2023-06-08T03:50:29.591028+0000","last_fullsized":"2023-06-08T03:50:29.591028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310861+0000","last_change":"2023-06-08T03:50:24.690477+0000","last_active":"2023-06-08T03:50:29.310861+0000","last_peered":"2023-06-08T03:50:29.310861+0000","last_clean":"2023-06-08T03:50:29.310861+0000","last_became_active":"2023-06-08T03:50:24.690125+0000","last_became_peered":"2023-06-08T03:50:24.690125+0000","last_unstale":"2023-06-08T03:50:29.310861+0000","last_undegraded":"2023-06-08T03:50:29.310861+0000","last_fullsized":"2023-06-08T03:50:29.310861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:40.966384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311958+0000","last_change":"2023-06-08T03:50:24.682936+0000","last_active":"2023-06-08T03:50:29.311958+0000","last_peered":"2023-06-08T03:50:29.311958+0000","last_clean":"2023-06-08T03:50:29.311958+0000","last_became_active":"2023-06-08T03:50:24.681545+0000","last_became_peered":"2023-06-08T03:50:24.681545+0000","last_unstale":"2023-06-08T03:50:29.311958+0000","last_undegraded":"2023-06-08T03:50:29.311958+0000","last_fullsized":"2023-06-08T03:50:29.311958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4451,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":270336,"data_stored":109499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":110083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":109555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":113062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":880640,"data_stored":703534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":700359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-08T03:50:43.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-08T03:50:43.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.177+0000 7f69ed7e7700 1 Processor -- start 2023-06-08T03:50:43.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.177+0000 7f69ed7e7700 1 -- start start 2023-06-08T03:50:43.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69ed7e7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 0x7f69e815c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69ed7e7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f69e806ae80 0x7f69e80ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69ed7e7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e80b20f0 con 0x7f69e80d2420 2023-06-08T03:50:43.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69ed7e7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e806b310 con 0x7f69e806ae80 2023-06-08T03:50:43.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69ed7e7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e806b5f0 con 0x7f69e80b1d20 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f69e80d2420 0x7f69e806c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58352/0 (socket says 172.21.15.79:58352) 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e77fe700 1 -- 172.21.15.79:0/1173827656 learned_addr learned my addr 172.21.15.79:0/1173827656 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659585698 0 0) 0x7f69e80b20f0 con 0x7f69e80d2420 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d00035b0 con 0x7f69e80d2420 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e67fc700 1 --2- 172.21.15.79:0/1173827656 >> v2:172.21.15.156:3300/0 conn(0x7f69e806ae80 0x7f69e80ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 0x7f69e815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295221954 0 0) 0x7f69d00035b0 con 0x7f69e80d2420 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 msgr2=0x7f69e815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 0x7f69e815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 >> v2:172.21.15.156:3300/0 conn(0x7f69e806ae80 msgr2=0x7f69e80ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 --2- 172.21.15.79:0/1173827656 >> v2:172.21.15.156:3300/0 conn(0x7f69e806ae80 0x7f69e80ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e806b8d0 con 0x7f69e80d2420 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f69e00019f0 con 0x7f69e80d2420 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 >> v1:172.21.15.79:6789/0 conn(0x7f69e80d2420 legacy=0x7f69e806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.178+0000 7f69e5ffb700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69d0003a70 0x7f69d0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0004850 0x7f69d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 0x7f69d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e806b310 con 0x7f69d0003a70 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e806b5f0 con 0x7f69d0004850 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d000ba60 con 0x7f69e80b1d20 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69d0003a70 0x7f69d0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e67fc700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0004850 0x7f69d0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e77fe700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 0x7f69d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 -- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 msgr2=0x7f69d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e80b1d20 0x7f69d0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 -- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0004850 msgr2=0x7f69d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0004850 0x7f69d0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 -- 172.21.15.79:0/1173827656 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00035b0 con 0x7f69d0003a70 2023-06-08T03:50:43.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e67fc700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0004850 0x7f69d0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69d0003a70 0x7f69d0003e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0098c0 tx=0x7f69d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38f6b73eb7482f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc00c070 con 0x7f69d0003a70 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc013610 con 0x7f69d0003a70 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.179+0000 7f69e5ffb700 1 -- 172.21.15.79:0/1173827656 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0139a0 con 0x7f69d0003a70 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1173827656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69d0003a70 msgr2=0x7f69d0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1173827656 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69d0003a70 0x7f69d0003e80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0098c0 tx=0x7f69d0005880 comp rx=0 tx=0).stop 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69e77fe700 1 -- 172.21.15.79:0/1173827656 reap_dead start 2023-06-08T03:50:43.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1173827656 shutdown_connections 2023-06-08T03:50:43.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1173827656 >> 172.21.15.79:0/1173827656 conn(0x7f69e809bbb0 msgr2=0x7f69e80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:43.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1173827656 shutdown_connections 2023-06-08T03:50:43.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.180+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1173827656 wait complete. 2023-06-08T03:50:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.181+0000 7f69ed7e7700 1 Processor -- start 2023-06-08T03:50:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.181+0000 7f69ed7e7700 1 -- start start 2023-06-08T03:50:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69ed7e7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0003a70 0x7f69e8154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69ed7e7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e8155da0 0x7f69e8156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69ed7e7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 0x7f69e80b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69ed7e7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e8154750 con 0x7f69e80b6cc0 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69ed7e7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80b7680 con 0x7f69d0003a70 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69ed7e7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81f1b20 con 0x7f69e8155da0 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 0x7f69e80b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 0x7f69e80b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34888/0 (socket says 172.21.15.79:34888) 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 -- 172.21.15.79:0/1459620371 learned_addr learned my addr 172.21.15.79:0/1459620371 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e67fc700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e8155da0 0x7f69e8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0003a70 0x7f69e8154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 -- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e8155da0 msgr2=0x7f69e8156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e8155da0 0x7f69e8156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 -- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0003a70 msgr2=0x7f69e8154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0003a70 0x7f69e8154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 -- 172.21.15.79:0/1459620371 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e8156b40 con 0x7f69e80b6cc0 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.182+0000 7f69e77fe700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 0x7f69e80b7140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69e0001330 tx=0x7f69e0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.183+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0017070 con 0x7f69e80b6cc0 2023-06-08T03:50:43.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.183+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e000eb50 con 0x7f69e80b6cc0 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.183+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e8156860 con 0x7f69e80b6cc0 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.183+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000ed00 con 0x7f69e80b6cc0 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.183+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e80c77d0 con 0x7f69e80b6cc0 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.184+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e8073e90 con 0x7f69e80b6cc0 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.184+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69e0013420 con 0x7f69e80b6cc0 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.184+0000 7f69cf7fe700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f69d406ace0 0x7f69d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.184+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f69e0089350 con 0x7f69e80b6cc0 2023-06-08T03:50:43.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.185+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f69d406ace0 0x7f69d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.185+0000 7f69e6ffd700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f69d406ace0 0x7f69d406d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69dc000c00 tx=0x7f69dc021040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.189+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f69e0054110 con 0x7f69e80b6cc0 2023-06-08T03:50:43.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.372+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 --> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69e80c7370 con 0x7f69d406ace0 2023-06-08T03:50:43.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.382+0000 7f69cf7fe700 1 -- 172.21.15.79:0/1459620371 <== mgr.4099 v2:172.21.15.156:6832/2537078349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367559 (secure 0 0 0) 0x7f69dc004040 con 0x7f69d406ace0 2023-06-08T03:50:43.389 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:43.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f69d406ace0 msgr2=0x7f69d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f69d406ace0 0x7f69d406d190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69dc000c00 tx=0x7f69dc021040 comp rx=0 tx=0).stop 2023-06-08T03:50:43.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 msgr2=0x7f69e80b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 0x7f69e80b7140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69e0001330 tx=0x7f69e0001ae0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 shutdown_connections 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69e80b6cc0 0x7f69e80b7140 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69d0003a70 0x7f69e8154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f69d406ace0 0x7f69d406d190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 --2- 172.21.15.79:0/1459620371 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69e8155da0 0x7f69e8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 >> 172.21.15.79:0/1459620371 conn(0x7f69e809bbb0 msgr2=0x7f69e80b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:43.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 shutdown_connections 2023-06-08T03:50:43.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.385+0000 7f69ed7e7700 1 -- 172.21.15.79:0/1459620371 wait complete. 2023-06-08T03:50:43.393 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-08T03:50:43.401 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-06-08T03:50:41.873159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594819,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218920,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755800,"statfs":{"total":773094113280,"available":772869939200,"internally_reserved":0,"allocated":3383296,"data_stored":2064910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312366+0000","last_change":"2023-06-08T03:50:24.695168+0000","last_active":"2023-06-08T03:50:29.312366+0000","last_peered":"2023-06-08T03:50:29.312366+0000","last_clean":"2023-06-08T03:50:29.312366+0000","last_became_active":"2023-06-08T03:50:24.694872+0000","last_became_peered":"2023-06-08T03:50:24.694872+0000","last_unstale":"2023-06-08T03:50:29.312366+0000","last_undegraded":"2023-06-08T03:50:29.312366+0000","last_fullsized":"2023-06-08T03:50:29.312366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791005+0000","last_change":"2023-06-08T03:50:24.687043+0000","last_active":"2023-06-08T03:50:29.791005+0000","last_peered":"2023-06-08T03:50:29.791005+0000","last_clean":"2023-06-08T03:50:29.791005+0000","last_became_active":"2023-06-08T03:50:24.686710+0000","last_became_peered":"2023-06-08T03:50:24.686710+0000","last_unstale":"2023-06-08T03:50:29.791005+0000","last_undegraded":"2023-06-08T03:50:29.791005+0000","last_fullsized":"2023-06-08T03:50:29.791005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591260+0000","last_change":"2023-06-08T03:50:24.686759+0000","last_active":"2023-06-08T03:50:29.591260+0000","last_peered":"2023-06-08T03:50:29.591260+0000","last_clean":"2023-06-08T03:50:29.591260+0000","last_became_active":"2023-06-08T03:50:24.686497+0000","last_became_peered":"2023-06-08T03:50:24.686497+0000","last_unstale":"2023-06-08T03:50:29.591260+0000","last_undegraded":"2023-06-08T03:50:29.591260+0000","last_fullsized":"2023-06-08T03:50:29.591260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:14.651224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591320+0000","last_change":"2023-06-08T03:50:24.691468+0000","last_active":"2023-06-08T03:50:29.591320+0000","last_peered":"2023-06-08T03:50:29.591320+0000","last_clean":"2023-06-08T03:50:29.591320+0000","last_became_active":"2023-06-08T03:50:24.691218+0000","last_became_peered":"2023-06-08T03:50:24.691218+0000","last_unstale":"2023-06-08T03:50:29.591320+0000","last_undegraded":"2023-06-08T03:50:29.591320+0000","last_fullsized":"2023-06-08T03:50:29.591320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309624+0000","last_change":"2023-06-08T03:50:24.685604+0000","last_active":"2023-06-08T03:50:29.309624+0000","last_peered":"2023-06-08T03:50:29.309624+0000","last_clean":"2023-06-08T03:50:29.309624+0000","last_became_active":"2023-06-08T03:50:24.685184+0000","last_became_peered":"2023-06-08T03:50:24.685184+0000","last_unstale":"2023-06-08T03:50:29.309624+0000","last_undegraded":"2023-06-08T03:50:29.309624+0000","last_fullsized":"2023-06-08T03:50:29.309624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:09:46.870511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789772+0000","last_change":"2023-06-08T03:50:24.698386+0000","last_active":"2023-06-08T03:50:29.789772+0000","last_peered":"2023-06-08T03:50:29.789772+0000","last_clean":"2023-06-08T03:50:29.789772+0000","last_became_active":"2023-06-08T03:50:24.698086+0000","last_became_peered":"2023-06-08T03:50:24.698086+0000","last_unstale":"2023-06-08T03:50:29.789772+0000","last_undegraded":"2023-06-08T03:50:29.789772+0000","last_fullsized":"2023-06-08T03:50:29.789772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:11.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790111+0000","last_change":"2023-06-08T03:50:24.693046+0000","last_active":"2023-06-08T03:50:29.790111+0000","last_peered":"2023-06-08T03:50:29.790111+0000","last_clean":"2023-06-08T03:50:29.790111+0000","last_became_active":"2023-06-08T03:50:24.692581+0000","last_became_peered":"2023-06-08T03:50:24.692581+0000","last_unstale":"2023-06-08T03:50:29.790111+0000","last_undegraded":"2023-06-08T03:50:29.790111+0000","last_fullsized":"2023-06-08T03:50:29.790111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:39.113365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790425+0000","last_change":"2023-06-08T03:50:24.689352+0000","last_active":"2023-06-08T03:50:29.790425+0000","last_peered":"2023-06-08T03:50:29.790425+0000","last_clean":"2023-06-08T03:50:29.790425+0000","last_became_active":"2023-06-08T03:50:24.688961+0000","last_became_peered":"2023-06-08T03:50:24.688961+0000","last_unstale":"2023-06-08T03:50:29.790425+0000","last_undegraded":"2023-06-08T03:50:29.790425+0000","last_fullsized":"2023-06-08T03:50:29.790425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:59:48.614865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310084+0000","last_change":"2023-06-08T03:50:24.682755+0000","last_active":"2023-06-08T03:50:29.310084+0000","last_peered":"2023-06-08T03:50:29.310084+0000","last_clean":"2023-06-08T03:50:29.310084+0000","last_became_active":"2023-06-08T03:50:24.682417+0000","last_became_peered":"2023-06-08T03:50:24.682417+0000","last_unstale":"2023-06-08T03:50:29.310084+0000","last_undegraded":"2023-06-08T03:50:29.310084+0000","last_fullsized":"2023-06-08T03:50:29.310084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:38:17.580156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791038+0000","last_change":"2023-06-08T03:50:24.697828+0000","last_active":"2023-06-08T03:50:29.791038+0000","last_peered":"2023-06-08T03:50:29.791038+0000","last_clean":"2023-06-08T03:50:29.791038+0000","last_became_active":"2023-06-08T03:50:24.697576+0000","last_became_peered":"2023-06-08T03:50:24.697576+0000","last_unstale":"2023-06-08T03:50:29.791038+0000","last_undegraded":"2023-06-08T03:50:29.791038+0000","last_fullsized":"2023-06-08T03:50:29.791038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789439+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.789439+0000","last_peered":"2023-06-08T03:50:29.789439+0000","last_clean":"2023-06-08T03:50:29.789439+0000","last_became_active":"2023-06-08T03:50:24.688197+0000","last_became_peered":"2023-06-08T03:50:24.688197+0000","last_unstale":"2023-06-08T03:50:29.789439+0000","last_undegraded":"2023-06-08T03:50:29.789439+0000","last_fullsized":"2023-06-08T03:50:29.789439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:04:27.557616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311848+0000","last_change":"2023-06-08T03:50:24.692867+0000","last_active":"2023-06-08T03:50:29.311848+0000","last_peered":"2023-06-08T03:50:29.311848+0000","last_clean":"2023-06-08T03:50:29.311848+0000","last_became_active":"2023-06-08T03:50:24.692613+0000","last_became_peered":"2023-06-08T03:50:24.692613+0000","last_unstale":"2023-06-08T03:50:29.311848+0000","last_undegraded":"2023-06-08T03:50:29.311848+0000","last_fullsized":"2023-06-08T03:50:29.311848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:02.122906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309156+0000","last_change":"2023-06-08T03:50:24.694092+0000","last_active":"2023-06-08T03:50:29.309156+0000","last_peered":"2023-06-08T03:50:29.309156+0000","last_clean":"2023-06-08T03:50:29.309156+0000","last_became_active":"2023-06-08T03:50:24.693783+0000","last_became_peered":"2023-06-08T03:50:24.693783+0000","last_unstale":"2023-06-08T03:50:29.309156+0000","last_undegraded":"2023-06-08T03:50:29.309156+0000","last_fullsized":"2023-06-08T03:50:29.309156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790036+0000","last_change":"2023-06-08T03:50:24.702721+0000","last_active":"2023-06-08T03:50:29.790036+0000","last_peered":"2023-06-08T03:50:29.790036+0000","last_clean":"2023-06-08T03:50:29.790036+0000","last_became_active":"2023-06-08T03:50:24.702438+0000","last_became_peered":"2023-06-08T03:50:24.702438+0000","last_unstale":"2023-06-08T03:50:29.790036+0000","last_undegraded":"2023-06-08T03:50:29.790036+0000","last_fullsized":"2023-06-08T03:50:29.790036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:59:10.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.692560+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.692035+0000","last_became_peered":"2023-06-08T03:50:24.692035+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790991+0000","last_change":"2023-06-08T03:50:24.698511+0000","last_active":"2023-06-08T03:50:29.790991+0000","last_peered":"2023-06-08T03:50:29.790991+0000","last_clean":"2023-06-08T03:50:29.790991+0000","last_became_active":"2023-06-08T03:50:24.698281+0000","last_became_peered":"2023-06-08T03:50:24.698281+0000","last_unstale":"2023-06-08T03:50:29.790991+0000","last_undegraded":"2023-06-08T03:50:29.790991+0000","last_fullsized":"2023-06-08T03:50:29.790991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:27:46.498562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789941+0000","last_change":"2023-06-08T03:50:24.695969+0000","last_active":"2023-06-08T03:50:29.789941+0000","last_peered":"2023-06-08T03:50:29.789941+0000","last_clean":"2023-06-08T03:50:29.789941+0000","last_became_active":"2023-06-08T03:50:24.695711+0000","last_became_peered":"2023-06-08T03:50:24.695711+0000","last_unstale":"2023-06-08T03:50:29.789941+0000","last_undegraded":"2023-06-08T03:50:29.789941+0000","last_fullsized":"2023-06-08T03:50:29.789941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:45:45.278948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591321+0000","last_change":"2023-06-08T03:50:24.690923+0000","last_active":"2023-06-08T03:50:29.591321+0000","last_peered":"2023-06-08T03:50:29.591321+0000","last_clean":"2023-06-08T03:50:29.591321+0000","last_became_active":"2023-06-08T03:50:24.690667+0000","last_became_peered":"2023-06-08T03:50:24.690667+0000","last_unstale":"2023-06-08T03:50:29.591321+0000","last_undegraded":"2023-06-08T03:50:29.591321+0000","last_fullsized":"2023-06-08T03:50:29.591321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311961+0000","last_change":"2023-06-08T03:50:24.701274+0000","last_active":"2023-06-08T03:50:29.311961+0000","last_peered":"2023-06-08T03:50:29.311961+0000","last_clean":"2023-06-08T03:50:29.311961+0000","last_became_active":"2023-06-08T03:50:24.700991+0000","last_became_peered":"2023-06-08T03:50:24.700991+0000","last_unstale":"2023-06-08T03:50:29.311961+0000","last_undegraded":"2023-06-08T03:50:29.311961+0000","last_fullsized":"2023-06-08T03:50:29.311961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789321+0000","last_change":"2023-06-08T03:50:24.690034+0000","last_active":"2023-06-08T03:50:29.789321+0000","last_peered":"2023-06-08T03:50:29.789321+0000","last_clean":"2023-06-08T03:50:29.789321+0000","last_became_active":"2023-06-08T03:50:24.689524+0000","last_became_peered":"2023-06-08T03:50:24.689524+0000","last_unstale":"2023-06-08T03:50:29.789321+0000","last_undegraded":"2023-06-08T03:50:29.789321+0000","last_fullsized":"2023-06-08T03:50:29.789321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790401+0000","last_change":"2023-06-08T03:50:24.702598+0000","last_active":"2023-06-08T03:50:29.790401+0000","last_peered":"2023-06-08T03:50:29.790401+0000","last_clean":"2023-06-08T03:50:29.790401+0000","last_became_active":"2023-06-08T03:50:24.702337+0000","last_became_peered":"2023-06-08T03:50:24.702337+0000","last_unstale":"2023-06-08T03:50:29.790401+0000","last_undegraded":"2023-06-08T03:50:29.790401+0000","last_fullsized":"2023-06-08T03:50:29.790401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:29:18.553774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789291+0000","last_change":"2023-06-08T03:50:24.689664+0000","last_active":"2023-06-08T03:50:29.789291+0000","last_peered":"2023-06-08T03:50:29.789291+0000","last_clean":"2023-06-08T03:50:29.789291+0000","last_became_active":"2023-06-08T03:50:24.689386+0000","last_became_peered":"2023-06-08T03:50:24.689386+0000","last_unstale":"2023-06-08T03:50:29.789291+0000","last_undegraded":"2023-06-08T03:50:29.789291+0000","last_fullsized":"2023-06-08T03:50:29.789291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:23:34.826399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308547+0000","last_change":"2023-06-08T03:50:24.690551+0000","last_active":"2023-06-08T03:50:29.308547+0000","last_peered":"2023-06-08T03:50:29.308547+0000","last_clean":"2023-06-08T03:50:29.308547+0000","last_became_active":"2023-06-08T03:50:24.689459+0000","last_became_peered":"2023-06-08T03:50:24.689459+0000","last_unstale":"2023-06-08T03:50:29.308547+0000","last_undegraded":"2023-06-08T03:50:29.308547+0000","last_fullsized":"2023-06-08T03:50:29.308547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:21:13.898688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789394+0000","last_change":"2023-06-08T03:50:24.702787+0000","last_active":"2023-06-08T03:50:29.789394+0000","last_peered":"2023-06-08T03:50:29.789394+0000","last_clean":"2023-06-08T03:50:29.789394+0000","last_became_active":"2023-06-08T03:50:24.702390+0000","last_became_peered":"2023-06-08T03:50:24.702390+0000","last_unstale":"2023-06-08T03:50:29.789394+0000","last_undegraded":"2023-06-08T03:50:29.789394+0000","last_fullsized":"2023-06-08T03:50:29.789394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310288+0000","last_change":"2023-06-08T03:50:24.689083+0000","last_active":"2023-06-08T03:50:29.310288+0000","last_peered":"2023-06-08T03:50:29.310288+0000","last_clean":"2023-06-08T03:50:29.310288+0000","last_became_active":"2023-06-08T03:50:24.688803+0000","last_became_peered":"2023-06-08T03:50:24.688803+0000","last_unstale":"2023-06-08T03:50:29.310288+0000","last_undegraded":"2023-06-08T03:50:29.310288+0000","last_fullsized":"2023-06-08T03:50:29.310288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:59:04.642632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789549+0000","last_change":"2023-06-08T03:50:24.689752+0000","last_active":"2023-06-08T03:50:29.789549+0000","last_peered":"2023-06-08T03:50:29.789549+0000","last_clean":"2023-06-08T03:50:29.789549+0000","last_became_active":"2023-06-08T03:50:24.689199+0000","last_became_peered":"2023-06-08T03:50:24.689199+0000","last_unstale":"2023-06-08T03:50:29.789549+0000","last_undegraded":"2023-06-08T03:50:29.789549+0000","last_fullsized":"2023-06-08T03:50:29.789549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:06:49.219275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790465+0000","last_change":"2023-06-08T03:50:24.699485+0000","last_active":"2023-06-08T03:50:29.790465+0000","last_peered":"2023-06-08T03:50:29.790465+0000","last_clean":"2023-06-08T03:50:29.790465+0000","last_became_active":"2023-06-08T03:50:24.699155+0000","last_became_peered":"2023-06-08T03:50:24.699155+0000","last_unstale":"2023-06-08T03:50:29.790465+0000","last_undegraded":"2023-06-08T03:50:29.790465+0000","last_fullsized":"2023-06-08T03:50:29.790465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:09:23.122980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790702+0000","last_change":"2023-06-08T03:50:28.917312+0000","last_active":"2023-06-08T03:50:29.790702+0000","last_peered":"2023-06-08T03:50:29.790702+0000","last_clean":"2023-06-08T03:50:29.790702+0000","last_became_active":"2023-06-08T03:50:28.916901+0000","last_became_peered":"2023-06-08T03:50:28.916901+0000","last_unstale":"2023-06-08T03:50:29.790702+0000","last_undegraded":"2023-06-08T03:50:29.790702+0000","last_fullsized":"2023-06-08T03:50:29.790702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6,0],"acting":[2,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789907+0000","last_change":"2023-06-08T03:50:24.285058+0000","last_active":"2023-06-08T03:50:29.789907+0000","last_peered":"2023-06-08T03:50:29.789907+0000","last_clean":"2023-06-08T03:50:29.789907+0000","last_became_active":"2023-06-08T03:50:24.284770+0000","last_became_peered":"2023-06-08T03:50:24.284770+0000","last_unstale":"2023-06-08T03:50:29.789907+0000","last_undegraded":"2023-06-08T03:50:29.789907+0000","last_fullsized":"2023-06-08T03:50:29.789907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:16:20.267305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789601+0000","last_change":"2023-06-08T03:50:24.698385+0000","last_active":"2023-06-08T03:50:29.789601+0000","last_peered":"2023-06-08T03:50:29.789601+0000","last_clean":"2023-06-08T03:50:29.789601+0000","last_became_active":"2023-06-08T03:50:24.697966+0000","last_became_peered":"2023-06-08T03:50:24.697966+0000","last_unstale":"2023-06-08T03:50:29.789601+0000","last_undegraded":"2023-06-08T03:50:29.789601+0000","last_fullsized":"2023-06-08T03:50:29.789601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:20.947165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791158+0000","last_change":"2023-06-08T03:50:28.922104+0000","last_active":"2023-06-08T03:50:29.791158+0000","last_peered":"2023-06-08T03:50:29.791158+0000","last_clean":"2023-06-08T03:50:29.791158+0000","last_became_active":"2023-06-08T03:50:28.921721+0000","last_became_peered":"2023-06-08T03:50:28.921721+0000","last_unstale":"2023-06-08T03:50:29.791158+0000","last_undegraded":"2023-06-08T03:50:29.791158+0000","last_fullsized":"2023-06-08T03:50:29.791158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:28.031781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,5],"acting":[2,7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308030+0000","last_change":"2023-06-08T03:50:24.286192+0000","last_active":"2023-06-08T03:50:29.308030+0000","last_peered":"2023-06-08T03:50:29.308030+0000","last_clean":"2023-06-08T03:50:29.308030+0000","last_became_active":"2023-06-08T03:50:24.285939+0000","last_became_peered":"2023-06-08T03:50:24.285939+0000","last_unstale":"2023-06-08T03:50:29.308030+0000","last_undegraded":"2023-06-08T03:50:29.308030+0000","last_fullsized":"2023-06-08T03:50:29.308030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:32.185279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308525+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.308525+0000","last_peered":"2023-06-08T03:50:29.308525+0000","last_clean":"2023-06-08T03:50:29.308525+0000","last_became_active":"2023-06-08T03:50:24.688036+0000","last_became_peered":"2023-06-08T03:50:24.688036+0000","last_unstale":"2023-06-08T03:50:29.308525+0000","last_undegraded":"2023-06-08T03:50:29.308525+0000","last_fullsized":"2023-06-08T03:50:29.308525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:23:40.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309722+0000","last_change":"2023-06-08T03:50:28.921813+0000","last_active":"2023-06-08T03:50:29.309722+0000","last_peered":"2023-06-08T03:50:29.309722+0000","last_clean":"2023-06-08T03:50:29.309722+0000","last_became_active":"2023-06-08T03:50:28.921392+0000","last_became_peered":"2023-06-08T03:50:28.921392+0000","last_unstale":"2023-06-08T03:50:29.309722+0000","last_undegraded":"2023-06-08T03:50:29.309722+0000","last_fullsized":"2023-06-08T03:50:29.309722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:18.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2,7],"acting":[6,3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310612+0000","last_change":"2023-06-08T03:50:24.281038+0000","last_active":"2023-06-08T03:50:29.310612+0000","last_peered":"2023-06-08T03:50:29.310612+0000","last_clean":"2023-06-08T03:50:29.310612+0000","last_became_active":"2023-06-08T03:50:24.280771+0000","last_became_peered":"2023-06-08T03:50:24.280771+0000","last_unstale":"2023-06-08T03:50:29.310612+0000","last_undegraded":"2023-06-08T03:50:29.310612+0000","last_fullsized":"2023-06-08T03:50:29.310612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:43.269177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790403+0000","last_change":"2023-06-08T03:50:24.686123+0000","last_active":"2023-06-08T03:50:29.790403+0000","last_peered":"2023-06-08T03:50:29.790403+0000","last_clean":"2023-06-08T03:50:29.790403+0000","last_became_active":"2023-06-08T03:50:24.685951+0000","last_became_peered":"2023-06-08T03:50:24.685951+0000","last_unstale":"2023-06-08T03:50:29.790403+0000","last_undegraded":"2023-06-08T03:50:29.790403+0000","last_fullsized":"2023-06-08T03:50:29.790403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:44:16.167507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789665+0000","last_change":"2023-06-08T03:50:28.929521+0000","last_active":"2023-06-08T03:50:29.789665+0000","last_peered":"2023-06-08T03:50:29.789665+0000","last_clean":"2023-06-08T03:50:29.789665+0000","last_became_active":"2023-06-08T03:50:28.929194+0000","last_became_peered":"2023-06-08T03:50:28.929194+0000","last_unstale":"2023-06-08T03:50:29.789665+0000","last_undegraded":"2023-06-08T03:50:29.789665+0000","last_fullsized":"2023-06-08T03:50:29.789665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:19:17.680617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,5],"acting":[3,2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312228+0000","last_change":"2023-06-08T03:50:24.278075+0000","last_active":"2023-06-08T03:50:29.312228+0000","last_peered":"2023-06-08T03:50:29.312228+0000","last_clean":"2023-06-08T03:50:29.312228+0000","last_became_active":"2023-06-08T03:50:24.277802+0000","last_became_peered":"2023-06-08T03:50:24.277802+0000","last_unstale":"2023-06-08T03:50:29.312228+0000","last_undegraded":"2023-06-08T03:50:29.312228+0000","last_fullsized":"2023-06-08T03:50:29.312228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789217+0000","last_change":"2023-06-08T03:50:24.699630+0000","last_active":"2023-06-08T03:50:29.789217+0000","last_peered":"2023-06-08T03:50:29.789217+0000","last_clean":"2023-06-08T03:50:29.789217+0000","last_became_active":"2023-06-08T03:50:24.699373+0000","last_became_peered":"2023-06-08T03:50:24.699373+0000","last_unstale":"2023-06-08T03:50:29.789217+0000","last_undegraded":"2023-06-08T03:50:29.789217+0000","last_fullsized":"2023-06-08T03:50:29.789217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:08:53.776951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789066+0000","last_change":"2023-06-08T03:50:28.923351+0000","last_active":"2023-06-08T03:50:29.789066+0000","last_peered":"2023-06-08T03:50:29.789066+0000","last_clean":"2023-06-08T03:50:29.789066+0000","last_became_active":"2023-06-08T03:50:28.922947+0000","last_became_peered":"2023-06-08T03:50:28.922947+0000","last_unstale":"2023-06-08T03:50:29.789066+0000","last_undegraded":"2023-06-08T03:50:29.789066+0000","last_fullsized":"2023-06-08T03:50:29.789066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:19:30.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,4],"acting":[2,1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790604+0000","last_change":"2023-06-08T03:50:24.276888+0000","last_active":"2023-06-08T03:50:29.790604+0000","last_peered":"2023-06-08T03:50:29.790604+0000","last_clean":"2023-06-08T03:50:29.790604+0000","last_became_active":"2023-06-08T03:50:24.276608+0000","last_became_peered":"2023-06-08T03:50:24.276608+0000","last_unstale":"2023-06-08T03:50:29.790604+0000","last_undegraded":"2023-06-08T03:50:29.790604+0000","last_fullsized":"2023-06-08T03:50:29.790604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:23.615413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307879+0000","last_change":"2023-06-08T03:50:24.690298+0000","last_active":"2023-06-08T03:50:29.307879+0000","last_peered":"2023-06-08T03:50:29.307879+0000","last_clean":"2023-06-08T03:50:29.307879+0000","last_became_active":"2023-06-08T03:50:24.689874+0000","last_became_peered":"2023-06-08T03:50:24.689874+0000","last_unstale":"2023-06-08T03:50:29.307879+0000","last_undegraded":"2023-06-08T03:50:29.307879+0000","last_fullsized":"2023-06-08T03:50:29.307879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309657+0000","last_change":"2023-06-08T03:50:28.925251+0000","last_active":"2023-06-08T03:50:29.309657+0000","last_peered":"2023-06-08T03:50:29.309657+0000","last_clean":"2023-06-08T03:50:29.309657+0000","last_became_active":"2023-06-08T03:50:28.923741+0000","last_became_peered":"2023-06-08T03:50:28.923741+0000","last_unstale":"2023-06-08T03:50:29.309657+0000","last_undegraded":"2023-06-08T03:50:29.309657+0000","last_fullsized":"2023-06-08T03:50:29.309657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:55:27.532871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7,1],"acting":[4,6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.285463+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.285152+0000","last_became_peered":"2023-06-08T03:50:24.285152+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308197+0000","last_change":"2023-06-08T03:50:24.689181+0000","last_active":"2023-06-08T03:50:29.308197+0000","last_peered":"2023-06-08T03:50:29.308197+0000","last_clean":"2023-06-08T03:50:29.308197+0000","last_became_active":"2023-06-08T03:50:24.688882+0000","last_became_peered":"2023-06-08T03:50:24.688882+0000","last_unstale":"2023-06-08T03:50:29.308197+0000","last_undegraded":"2023-06-08T03:50:29.308197+0000","last_fullsized":"2023-06-08T03:50:29.308197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:58:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308089+0000","last_change":"2023-06-08T03:50:28.919724+0000","last_active":"2023-06-08T03:50:29.308089+0000","last_peered":"2023-06-08T03:50:29.308089+0000","last_clean":"2023-06-08T03:50:29.308089+0000","last_became_active":"2023-06-08T03:50:28.919364+0000","last_became_peered":"2023-06-08T03:50:28.919364+0000","last_unstale":"2023-06-08T03:50:29.308089+0000","last_undegraded":"2023-06-08T03:50:29.308089+0000","last_fullsized":"2023-06-08T03:50:29.308089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:29.657933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4,3],"acting":[6,1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:33.549882+0000","last_change":"2023-06-08T03:50:24.284754+0000","last_active":"2023-06-08T03:50:33.549882+0000","last_peered":"2023-06-08T03:50:33.549882+0000","last_clean":"2023-06-08T03:50:33.549882+0000","last_became_active":"2023-06-08T03:50:24.281521+0000","last_became_peered":"2023-06-08T03:50:24.281521+0000","last_unstale":"2023-06-08T03:50:33.549882+0000","last_undegraded":"2023-06-08T03:50:33.549882+0000","last_fullsized":"2023-06-08T03:50:33.549882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:25.213919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3653,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:24.688053+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:24.687402+0000","last_became_peered":"2023-06-08T03:50:24.687402+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:02.669215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790804+0000","last_change":"2023-06-08T03:50:28.918485+0000","last_active":"2023-06-08T03:50:29.790804+0000","last_peered":"2023-06-08T03:50:29.790804+0000","last_clean":"2023-06-08T03:50:29.790804+0000","last_became_active":"2023-06-08T03:50:28.918103+0000","last_became_peered":"2023-06-08T03:50:28.918103+0000","last_unstale":"2023-06-08T03:50:29.790804+0000","last_undegraded":"2023-06-08T03:50:29.790804+0000","last_fullsized":"2023-06-08T03:50:29.790804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:26:43.524009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,6],"acting":[3,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789698+0000","last_change":"2023-06-08T03:50:24.287653+0000","last_active":"2023-06-08T03:50:29.789698+0000","last_peered":"2023-06-08T03:50:29.789698+0000","last_clean":"2023-06-08T03:50:29.789698+0000","last_became_active":"2023-06-08T03:50:24.287374+0000","last_became_peered":"2023-06-08T03:50:24.287374+0000","last_unstale":"2023-06-08T03:50:29.789698+0000","last_undegraded":"2023-06-08T03:50:29.789698+0000","last_fullsized":"2023-06-08T03:50:29.789698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:25:09.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308854+0000","last_change":"2023-06-08T03:50:24.693976+0000","last_active":"2023-06-08T03:50:29.308854+0000","last_peered":"2023-06-08T03:50:29.308854+0000","last_clean":"2023-06-08T03:50:29.308854+0000","last_became_active":"2023-06-08T03:50:24.693510+0000","last_became_peered":"2023-06-08T03:50:24.693510+0000","last_unstale":"2023-06-08T03:50:29.308854+0000","last_undegraded":"2023-06-08T03:50:29.308854+0000","last_fullsized":"2023-06-08T03:50:29.308854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:30.446775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310556+0000","last_change":"2023-06-08T03:50:28.925279+0000","last_active":"2023-06-08T03:50:29.310556+0000","last_peered":"2023-06-08T03:50:29.310556+0000","last_clean":"2023-06-08T03:50:29.310556+0000","last_became_active":"2023-06-08T03:50:28.923608+0000","last_became_peered":"2023-06-08T03:50:28.923608+0000","last_unstale":"2023-06-08T03:50:29.310556+0000","last_undegraded":"2023-06-08T03:50:29.310556+0000","last_fullsized":"2023-06-08T03:50:29.310556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:32.233874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5,7],"acting":[4,2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309210+0000","last_change":"2023-06-08T03:50:24.285047+0000","last_active":"2023-06-08T03:50:29.309210+0000","last_peered":"2023-06-08T03:50:29.309210+0000","last_clean":"2023-06-08T03:50:29.309210+0000","last_became_active":"2023-06-08T03:50:24.284768+0000","last_became_peered":"2023-06-08T03:50:24.284768+0000","last_unstale":"2023-06-08T03:50:29.309210+0000","last_undegraded":"2023-06-08T03:50:29.309210+0000","last_fullsized":"2023-06-08T03:50:29.309210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790899+0000","last_change":"2023-06-08T03:50:24.691967+0000","last_active":"2023-06-08T03:50:29.790899+0000","last_peered":"2023-06-08T03:50:29.790899+0000","last_clean":"2023-06-08T03:50:29.790899+0000","last_became_active":"2023-06-08T03:50:24.691699+0000","last_became_peered":"2023-06-08T03:50:24.691699+0000","last_unstale":"2023-06-08T03:50:29.790899+0000","last_undegraded":"2023-06-08T03:50:29.790899+0000","last_fullsized":"2023-06-08T03:50:29.790899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:40:30.017039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310163+0000","last_change":"2023-06-08T03:50:28.926277+0000","last_active":"2023-06-08T03:50:29.310163+0000","last_peered":"2023-06-08T03:50:29.310163+0000","last_clean":"2023-06-08T03:50:29.310163+0000","last_became_active":"2023-06-08T03:50:28.925921+0000","last_became_peered":"2023-06-08T03:50:28.925921+0000","last_unstale":"2023-06-08T03:50:29.310163+0000","last_undegraded":"2023-06-08T03:50:29.310163+0000","last_fullsized":"2023-06-08T03:50:29.310163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:32:27.673747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1,6],"acting":[4,5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789845+0000","last_change":"2023-06-08T03:50:24.283127+0000","last_active":"2023-06-08T03:50:29.789845+0000","last_peered":"2023-06-08T03:50:29.789845+0000","last_clean":"2023-06-08T03:50:29.789845+0000","last_became_active":"2023-06-08T03:50:24.282515+0000","last_became_peered":"2023-06-08T03:50:24.282515+0000","last_unstale":"2023-06-08T03:50:29.789845+0000","last_undegraded":"2023-06-08T03:50:29.789845+0000","last_fullsized":"2023-06-08T03:50:29.789845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:55.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311940+0000","last_change":"2023-06-08T03:50:24.687906+0000","last_active":"2023-06-08T03:50:29.311940+0000","last_peered":"2023-06-08T03:50:29.311940+0000","last_clean":"2023-06-08T03:50:29.311940+0000","last_became_active":"2023-06-08T03:50:24.687630+0000","last_became_peered":"2023-06-08T03:50:24.687630+0000","last_unstale":"2023-06-08T03:50:29.311940+0000","last_undegraded":"2023-06-08T03:50:29.311940+0000","last_fullsized":"2023-06-08T03:50:29.311940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:12.100669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790030+0000","last_change":"2023-06-08T03:50:28.918543+0000","last_active":"2023-06-08T03:50:29.790030+0000","last_peered":"2023-06-08T03:50:29.790030+0000","last_clean":"2023-06-08T03:50:29.790030+0000","last_became_active":"2023-06-08T03:50:28.918124+0000","last_became_peered":"2023-06-08T03:50:28.918124+0000","last_unstale":"2023-06-08T03:50:29.790030+0000","last_undegraded":"2023-06-08T03:50:29.790030+0000","last_fullsized":"2023-06-08T03:50:29.790030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:32:30.430868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4,0],"acting":[3,6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789271+0000","last_change":"2023-06-08T03:50:24.286716+0000","last_active":"2023-06-08T03:50:29.789271+0000","last_peered":"2023-06-08T03:50:29.789271+0000","last_clean":"2023-06-08T03:50:29.789271+0000","last_became_active":"2023-06-08T03:50:24.286096+0000","last_became_peered":"2023-06-08T03:50:24.286096+0000","last_unstale":"2023-06-08T03:50:29.789271+0000","last_undegraded":"2023-06-08T03:50:29.789271+0000","last_fullsized":"2023-06-08T03:50:29.789271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312273+0000","last_change":"2023-06-08T03:50:24.690535+0000","last_active":"2023-06-08T03:50:29.312273+0000","last_peered":"2023-06-08T03:50:29.312273+0000","last_clean":"2023-06-08T03:50:29.312273+0000","last_became_active":"2023-06-08T03:50:24.689649+0000","last_became_peered":"2023-06-08T03:50:24.689649+0000","last_unstale":"2023-06-08T03:50:29.312273+0000","last_undegraded":"2023-06-08T03:50:29.312273+0000","last_fullsized":"2023-06-08T03:50:29.312273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789697+0000","last_change":"2023-06-08T03:50:28.926759+0000","last_active":"2023-06-08T03:50:29.789697+0000","last_peered":"2023-06-08T03:50:29.789697+0000","last_clean":"2023-06-08T03:50:29.789697+0000","last_became_active":"2023-06-08T03:50:28.926391+0000","last_became_peered":"2023-06-08T03:50:28.926391+0000","last_unstale":"2023-06-08T03:50:29.789697+0000","last_undegraded":"2023-06-08T03:50:29.789697+0000","last_fullsized":"2023-06-08T03:50:29.789697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:28:31.793415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,7],"acting":[1,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.286445+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.286062+0000","last_became_peered":"2023-06-08T03:50:24.286062+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791098+0000","last_change":"2023-06-08T03:50:24.692233+0000","last_active":"2023-06-08T03:50:29.791098+0000","last_peered":"2023-06-08T03:50:29.791098+0000","last_clean":"2023-06-08T03:50:29.791098+0000","last_became_active":"2023-06-08T03:50:24.691584+0000","last_became_peered":"2023-06-08T03:50:24.691584+0000","last_unstale":"2023-06-08T03:50:29.791098+0000","last_undegraded":"2023-06-08T03:50:29.791098+0000","last_fullsized":"2023-06-08T03:50:29.791098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:43.387552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309967+0000","last_change":"2023-06-08T03:50:28.922207+0000","last_active":"2023-06-08T03:50:29.309967+0000","last_peered":"2023-06-08T03:50:29.309967+0000","last_clean":"2023-06-08T03:50:29.309967+0000","last_became_active":"2023-06-08T03:50:28.921809+0000","last_became_peered":"2023-06-08T03:50:28.921809+0000","last_unstale":"2023-06-08T03:50:29.309967+0000","last_undegraded":"2023-06-08T03:50:29.309967+0000","last_fullsized":"2023-06-08T03:50:29.309967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:23:47.506790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,0],"acting":[6,4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790800+0000","last_change":"2023-06-08T03:50:24.280168+0000","last_active":"2023-06-08T03:50:29.790800+0000","last_peered":"2023-06-08T03:50:29.790800+0000","last_clean":"2023-06-08T03:50:29.790800+0000","last_became_active":"2023-06-08T03:50:24.279947+0000","last_became_peered":"2023-06-08T03:50:24.279947+0000","last_unstale":"2023-06-08T03:50:29.790800+0000","last_undegraded":"2023-06-08T03:50:29.790800+0000","last_fullsized":"2023-06-08T03:50:29.790800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:11.091336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790420+0000","last_change":"2023-06-08T03:50:24.702952+0000","last_active":"2023-06-08T03:50:29.790420+0000","last_peered":"2023-06-08T03:50:29.790420+0000","last_clean":"2023-06-08T03:50:29.790420+0000","last_became_active":"2023-06-08T03:50:24.702213+0000","last_became_peered":"2023-06-08T03:50:24.702213+0000","last_unstale":"2023-06-08T03:50:29.790420+0000","last_undegraded":"2023-06-08T03:50:29.790420+0000","last_fullsized":"2023-06-08T03:50:29.790420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:57:59.540147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310770+0000","last_change":"2023-06-08T03:50:28.921768+0000","last_active":"2023-06-08T03:50:29.310770+0000","last_peered":"2023-06-08T03:50:29.310770+0000","last_clean":"2023-06-08T03:50:29.310770+0000","last_became_active":"2023-06-08T03:50:28.921477+0000","last_became_peered":"2023-06-08T03:50:28.921477+0000","last_unstale":"2023-06-08T03:50:29.310770+0000","last_undegraded":"2023-06-08T03:50:29.310770+0000","last_fullsized":"2023-06-08T03:50:29.310770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:06.300830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790697+0000","last_change":"2023-06-08T03:50:24.287775+0000","last_active":"2023-06-08T03:50:29.790697+0000","last_peered":"2023-06-08T03:50:29.790697+0000","last_clean":"2023-06-08T03:50:29.790697+0000","last_became_active":"2023-06-08T03:50:24.287136+0000","last_became_peered":"2023-06-08T03:50:24.287136+0000","last_unstale":"2023-06-08T03:50:29.790697+0000","last_undegraded":"2023-06-08T03:50:29.790697+0000","last_fullsized":"2023-06-08T03:50:29.790697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:17:17.207119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789501+0000","last_change":"2023-06-08T03:50:24.702234+0000","last_active":"2023-06-08T03:50:29.789501+0000","last_peered":"2023-06-08T03:50:29.789501+0000","last_clean":"2023-06-08T03:50:29.789501+0000","last_became_active":"2023-06-08T03:50:24.701881+0000","last_became_peered":"2023-06-08T03:50:24.701881+0000","last_unstale":"2023-06-08T03:50:29.789501+0000","last_undegraded":"2023-06-08T03:50:29.789501+0000","last_fullsized":"2023-06-08T03:50:29.789501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:52:37.709189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308237+0000","last_change":"2023-06-08T03:50:28.914578+0000","last_active":"2023-06-08T03:50:29.308237+0000","last_peered":"2023-06-08T03:50:29.308237+0000","last_clean":"2023-06-08T03:50:29.308237+0000","last_became_active":"2023-06-08T03:50:28.914064+0000","last_became_peered":"2023-06-08T03:50:28.914064+0000","last_unstale":"2023-06-08T03:50:29.308237+0000","last_undegraded":"2023-06-08T03:50:29.308237+0000","last_fullsized":"2023-06-08T03:50:29.308237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:52.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,7],"acting":[6,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.283466+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.283156+0000","last_became_peered":"2023-06-08T03:50:24.283156+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:25.016985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310840+0000","last_change":"2023-06-08T03:50:24.680665+0000","last_active":"2023-06-08T03:50:29.310840+0000","last_peered":"2023-06-08T03:50:29.310840+0000","last_clean":"2023-06-08T03:50:29.310840+0000","last_became_active":"2023-06-08T03:50:24.680424+0000","last_became_peered":"2023-06-08T03:50:24.680424+0000","last_unstale":"2023-06-08T03:50:29.310840+0000","last_undegraded":"2023-06-08T03:50:29.310840+0000","last_fullsized":"2023-06-08T03:50:29.310840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:24.624745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789596+0000","last_change":"2023-06-08T03:50:28.919899+0000","last_active":"2023-06-08T03:50:29.789596+0000","last_peered":"2023-06-08T03:50:29.789596+0000","last_clean":"2023-06-08T03:50:29.789596+0000","last_became_active":"2023-06-08T03:50:28.919515+0000","last_became_peered":"2023-06-08T03:50:28.919515+0000","last_unstale":"2023-06-08T03:50:29.789596+0000","last_undegraded":"2023-06-08T03:50:29.789596+0000","last_fullsized":"2023-06-08T03:50:29.789596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:49:21.771214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5,1],"acting":[7,0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789597+0000","last_change":"2023-06-08T03:50:24.282752+0000","last_active":"2023-06-08T03:50:29.789597+0000","last_peered":"2023-06-08T03:50:29.789597+0000","last_clean":"2023-06-08T03:50:29.789597+0000","last_became_active":"2023-06-08T03:50:24.282473+0000","last_became_peered":"2023-06-08T03:50:24.282473+0000","last_unstale":"2023-06-08T03:50:29.789597+0000","last_undegraded":"2023-06-08T03:50:29.789597+0000","last_fullsized":"2023-06-08T03:50:29.789597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:15:57.351189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591060+0000","last_change":"2023-06-08T03:50:24.691781+0000","last_active":"2023-06-08T03:50:29.591060+0000","last_peered":"2023-06-08T03:50:29.591060+0000","last_clean":"2023-06-08T03:50:29.591060+0000","last_became_active":"2023-06-08T03:50:24.690055+0000","last_became_peered":"2023-06-08T03:50:24.690055+0000","last_unstale":"2023-06-08T03:50:29.591060+0000","last_undegraded":"2023-06-08T03:50:29.591060+0000","last_fullsized":"2023-06-08T03:50:29.591060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311854+0000","last_change":"2023-06-08T03:50:28.927198+0000","last_active":"2023-06-08T03:50:29.311854+0000","last_peered":"2023-06-08T03:50:29.311854+0000","last_clean":"2023-06-08T03:50:29.311854+0000","last_became_active":"2023-06-08T03:50:28.926812+0000","last_became_peered":"2023-06-08T03:50:28.926812+0000","last_unstale":"2023-06-08T03:50:29.311854+0000","last_undegraded":"2023-06-08T03:50:29.311854+0000","last_fullsized":"2023-06-08T03:50:29.311854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4,2],"acting":[5,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789506+0000","last_change":"2023-06-08T03:50:24.287767+0000","last_active":"2023-06-08T03:50:29.789506+0000","last_peered":"2023-06-08T03:50:29.789506+0000","last_clean":"2023-06-08T03:50:29.789506+0000","last_became_active":"2023-06-08T03:50:24.287060+0000","last_became_peered":"2023-06-08T03:50:24.287060+0000","last_unstale":"2023-06-08T03:50:29.789506+0000","last_undegraded":"2023-06-08T03:50:29.789506+0000","last_fullsized":"2023-06-08T03:50:29.789506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:27.592926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790578+0000","last_change":"2023-06-08T03:50:24.700545+0000","last_active":"2023-06-08T03:50:29.790578+0000","last_peered":"2023-06-08T03:50:29.790578+0000","last_clean":"2023-06-08T03:50:29.790578+0000","last_became_active":"2023-06-08T03:50:24.700276+0000","last_became_peered":"2023-06-08T03:50:24.700276+0000","last_unstale":"2023-06-08T03:50:29.790578+0000","last_undegraded":"2023-06-08T03:50:29.790578+0000","last_fullsized":"2023-06-08T03:50:29.790578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:50.252975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791082+0000","last_change":"2023-06-08T03:50:28.920817+0000","last_active":"2023-06-08T03:50:29.791082+0000","last_peered":"2023-06-08T03:50:29.791082+0000","last_clean":"2023-06-08T03:50:29.791082+0000","last_became_active":"2023-06-08T03:50:28.920485+0000","last_became_peered":"2023-06-08T03:50:28.920485+0000","last_unstale":"2023-06-08T03:50:29.791082+0000","last_undegraded":"2023-06-08T03:50:29.791082+0000","last_fullsized":"2023-06-08T03:50:29.791082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:36.826227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,5],"acting":[1,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790009+0000","last_change":"2023-06-08T03:50:24.283331+0000","last_active":"2023-06-08T03:50:29.790009+0000","last_peered":"2023-06-08T03:50:29.790009+0000","last_clean":"2023-06-08T03:50:29.790009+0000","last_became_active":"2023-06-08T03:50:24.282856+0000","last_became_peered":"2023-06-08T03:50:24.282856+0000","last_unstale":"2023-06-08T03:50:29.790009+0000","last_undegraded":"2023-06-08T03:50:29.790009+0000","last_fullsized":"2023-06-08T03:50:29.790009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:52:45.337874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790917+0000","last_change":"2023-06-08T03:50:24.683795+0000","last_active":"2023-06-08T03:50:29.790917+0000","last_peered":"2023-06-08T03:50:29.790917+0000","last_clean":"2023-06-08T03:50:29.790917+0000","last_became_active":"2023-06-08T03:50:24.683510+0000","last_became_peered":"2023-06-08T03:50:24.683510+0000","last_unstale":"2023-06-08T03:50:29.790917+0000","last_undegraded":"2023-06-08T03:50:29.790917+0000","last_fullsized":"2023-06-08T03:50:29.790917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790162+0000","last_change":"2023-06-08T03:50:28.920202+0000","last_active":"2023-06-08T03:50:29.790162+0000","last_peered":"2023-06-08T03:50:29.790162+0000","last_clean":"2023-06-08T03:50:29.790162+0000","last_became_active":"2023-06-08T03:50:28.919960+0000","last_became_peered":"2023-06-08T03:50:28.919960+0000","last_unstale":"2023-06-08T03:50:29.790162+0000","last_undegraded":"2023-06-08T03:50:29.790162+0000","last_fullsized":"2023-06-08T03:50:29.790162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:44.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4,5],"acting":[3,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307873+0000","last_change":"2023-06-08T03:50:24.284333+0000","last_active":"2023-06-08T03:50:29.307873+0000","last_peered":"2023-06-08T03:50:29.307873+0000","last_clean":"2023-06-08T03:50:29.307873+0000","last_became_active":"2023-06-08T03:50:24.284039+0000","last_became_peered":"2023-06-08T03:50:24.284039+0000","last_unstale":"2023-06-08T03:50:29.307873+0000","last_undegraded":"2023-06-08T03:50:29.307873+0000","last_fullsized":"2023-06-08T03:50:29.307873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:37:13.194212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789927+0000","last_change":"2023-06-08T03:50:24.691444+0000","last_active":"2023-06-08T03:50:29.789927+0000","last_peered":"2023-06-08T03:50:29.789927+0000","last_clean":"2023-06-08T03:50:29.789927+0000","last_became_active":"2023-06-08T03:50:24.690971+0000","last_became_peered":"2023-06-08T03:50:24.690971+0000","last_unstale":"2023-06-08T03:50:29.789927+0000","last_undegraded":"2023-06-08T03:50:29.789927+0000","last_fullsized":"2023-06-08T03:50:29.789927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:38:23.297572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790270+0000","last_change":"2023-06-08T03:50:28.929890+0000","last_active":"2023-06-08T03:50:29.790270+0000","last_peered":"2023-06-08T03:50:29.790270+0000","last_clean":"2023-06-08T03:50:29.790270+0000","last_became_active":"2023-06-08T03:50:28.929496+0000","last_became_peered":"2023-06-08T03:50:28.929496+0000","last_unstale":"2023-06-08T03:50:29.790270+0000","last_undegraded":"2023-06-08T03:50:29.790270+0000","last_fullsized":"2023-06-08T03:50:29.790270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:21:51.680343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310379+0000","last_change":"2023-06-08T03:50:24.282724+0000","last_active":"2023-06-08T03:50:29.310379+0000","last_peered":"2023-06-08T03:50:29.310379+0000","last_clean":"2023-06-08T03:50:29.310379+0000","last_became_active":"2023-06-08T03:50:24.282451+0000","last_became_peered":"2023-06-08T03:50:24.282451+0000","last_unstale":"2023-06-08T03:50:29.310379+0000","last_undegraded":"2023-06-08T03:50:29.310379+0000","last_fullsized":"2023-06-08T03:50:29.310379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:44:39.775596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790625+0000","last_change":"2023-06-08T03:50:19.804887+0000","last_active":"2023-06-08T03:50:29.790625+0000","last_peered":"2023-06-08T03:50:29.790625+0000","last_clean":"2023-06-08T03:50:29.790625+0000","last_became_active":"2023-06-08T03:50:19.804546+0000","last_became_peered":"2023-06-08T03:50:19.804546+0000","last_unstale":"2023-06-08T03:50:29.790625+0000","last_undegraded":"2023-06-08T03:50:29.790625+0000","last_fullsized":"2023-06-08T03:50:29.790625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:34.859632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307694+0000","last_change":"2023-06-08T03:50:24.686843+0000","last_active":"2023-06-08T03:50:29.307694+0000","last_peered":"2023-06-08T03:50:29.307694+0000","last_clean":"2023-06-08T03:50:29.307694+0000","last_became_active":"2023-06-08T03:50:24.686587+0000","last_became_peered":"2023-06-08T03:50:24.686587+0000","last_unstale":"2023-06-08T03:50:29.307694+0000","last_undegraded":"2023-06-08T03:50:29.307694+0000","last_fullsized":"2023-06-08T03:50:29.307694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:36.573402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308557+0000","last_change":"2023-06-08T03:50:28.924976+0000","last_active":"2023-06-08T03:50:29.308557+0000","last_peered":"2023-06-08T03:50:29.308557+0000","last_clean":"2023-06-08T03:50:29.308557+0000","last_became_active":"2023-06-08T03:50:28.924335+0000","last_became_peered":"2023-06-08T03:50:28.924335+0000","last_unstale":"2023-06-08T03:50:29.308557+0000","last_undegraded":"2023-06-08T03:50:29.308557+0000","last_fullsized":"2023-06-08T03:50:29.308557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:47:44.099384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789646+0000","last_change":"2023-06-08T03:50:19.172271+0000","last_active":"2023-06-08T03:50:29.789646+0000","last_peered":"2023-06-08T03:50:29.789646+0000","last_clean":"2023-06-08T03:50:29.789646+0000","last_became_active":"2023-06-08T03:50:19.171917+0000","last_became_peered":"2023-06-08T03:50:19.171917+0000","last_unstale":"2023-06-08T03:50:29.789646+0000","last_undegraded":"2023-06-08T03:50:29.789646+0000","last_fullsized":"2023-06-08T03:50:29.789646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:01:56.709362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308387+0000","last_change":"2023-06-08T03:50:24.284253+0000","last_active":"2023-06-08T03:50:29.308387+0000","last_peered":"2023-06-08T03:50:29.308387+0000","last_clean":"2023-06-08T03:50:29.308387+0000","last_became_active":"2023-06-08T03:50:24.283958+0000","last_became_peered":"2023-06-08T03:50:24.283958+0000","last_unstale":"2023-06-08T03:50:29.308387+0000","last_undegraded":"2023-06-08T03:50:29.308387+0000","last_fullsized":"2023-06-08T03:50:29.308387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:02:34.479410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.690498+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.690242+0000","last_became_peered":"2023-06-08T03:50:24.690242+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:23:08.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790091+0000","last_change":"2023-06-08T03:50:28.921864+0000","last_active":"2023-06-08T03:50:29.790091+0000","last_peered":"2023-06-08T03:50:29.790091+0000","last_clean":"2023-06-08T03:50:29.790091+0000","last_became_active":"2023-06-08T03:50:28.921426+0000","last_became_peered":"2023-06-08T03:50:28.921426+0000","last_unstale":"2023-06-08T03:50:29.790091+0000","last_undegraded":"2023-06-08T03:50:29.790091+0000","last_fullsized":"2023-06-08T03:50:29.790091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:43:24.342082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,0],"acting":[3,7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309435+0000","last_change":"2023-06-08T03:50:19.171073+0000","last_active":"2023-06-08T03:50:29.309435+0000","last_peered":"2023-06-08T03:50:29.309435+0000","last_clean":"2023-06-08T03:50:29.309435+0000","last_became_active":"2023-06-08T03:50:19.170775+0000","last_became_peered":"2023-06-08T03:50:19.170775+0000","last_unstale":"2023-06-08T03:50:29.309435+0000","last_undegraded":"2023-06-08T03:50:29.309435+0000","last_fullsized":"2023-06-08T03:50:29.309435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:51:24.758705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791212+0000","last_change":"2023-06-08T03:50:19.167800+0000","last_active":"2023-06-08T03:50:29.791212+0000","last_peered":"2023-06-08T03:50:29.791212+0000","last_clean":"2023-06-08T03:50:29.791212+0000","last_became_active":"2023-06-08T03:50:19.167412+0000","last_became_peered":"2023-06-08T03:50:19.167412+0000","last_unstale":"2023-06-08T03:50:29.791212+0000","last_undegraded":"2023-06-08T03:50:29.791212+0000","last_fullsized":"2023-06-08T03:50:29.791212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:45:23.552252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309769+0000","last_change":"2023-06-08T03:50:24.280635+0000","last_active":"2023-06-08T03:50:29.309769+0000","last_peered":"2023-06-08T03:50:29.309769+0000","last_clean":"2023-06-08T03:50:29.309769+0000","last_became_active":"2023-06-08T03:50:24.280376+0000","last_became_peered":"2023-06-08T03:50:24.280376+0000","last_unstale":"2023-06-08T03:50:29.309769+0000","last_undegraded":"2023-06-08T03:50:29.309769+0000","last_fullsized":"2023-06-08T03:50:29.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:41.269585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312281+0000","last_change":"2023-06-08T03:50:24.685266+0000","last_active":"2023-06-08T03:50:29.312281+0000","last_peered":"2023-06-08T03:50:29.312281+0000","last_clean":"2023-06-08T03:50:29.312281+0000","last_became_active":"2023-06-08T03:50:24.684970+0000","last_became_peered":"2023-06-08T03:50:24.684970+0000","last_unstale":"2023-06-08T03:50:29.312281+0000","last_undegraded":"2023-06-08T03:50:29.312281+0000","last_fullsized":"2023-06-08T03:50:29.312281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789896+0000","last_change":"2023-06-08T03:50:28.926537+0000","last_active":"2023-06-08T03:50:29.789896+0000","last_peered":"2023-06-08T03:50:29.789896+0000","last_clean":"2023-06-08T03:50:29.789896+0000","last_became_active":"2023-06-08T03:50:28.926095+0000","last_became_peered":"2023-06-08T03:50:28.926095+0000","last_unstale":"2023-06-08T03:50:29.789896+0000","last_undegraded":"2023-06-08T03:50:29.789896+0000","last_fullsized":"2023-06-08T03:50:29.789896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:16:59.025169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,1],"acting":[7,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590877+0000","last_change":"2023-06-08T03:50:19.171834+0000","last_active":"2023-06-08T03:50:29.590877+0000","last_peered":"2023-06-08T03:50:29.590877+0000","last_clean":"2023-06-08T03:50:29.590877+0000","last_became_active":"2023-06-08T03:50:19.171541+0000","last_became_peered":"2023-06-08T03:50:19.171541+0000","last_unstale":"2023-06-08T03:50:29.590877+0000","last_undegraded":"2023-06-08T03:50:29.590877+0000","last_fullsized":"2023-06-08T03:50:29.590877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:42.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310150+0000","last_change":"2023-06-08T03:50:24.275533+0000","last_active":"2023-06-08T03:50:29.310150+0000","last_peered":"2023-06-08T03:50:29.310150+0000","last_clean":"2023-06-08T03:50:29.310150+0000","last_became_active":"2023-06-08T03:50:24.275086+0000","last_became_peered":"2023-06-08T03:50:24.275086+0000","last_unstale":"2023-06-08T03:50:29.310150+0000","last_undegraded":"2023-06-08T03:50:29.310150+0000","last_fullsized":"2023-06-08T03:50:29.310150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:49.514141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309093+0000","last_change":"2023-06-08T03:50:24.694692+0000","last_active":"2023-06-08T03:50:29.309093+0000","last_peered":"2023-06-08T03:50:29.309093+0000","last_clean":"2023-06-08T03:50:29.309093+0000","last_became_active":"2023-06-08T03:50:24.694404+0000","last_became_peered":"2023-06-08T03:50:24.694404+0000","last_unstale":"2023-06-08T03:50:29.309093+0000","last_undegraded":"2023-06-08T03:50:29.309093+0000","last_fullsized":"2023-06-08T03:50:29.309093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:06.873559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591398+0000","last_change":"2023-06-08T03:50:28.926999+0000","last_active":"2023-06-08T03:50:29.591398+0000","last_peered":"2023-06-08T03:50:29.591398+0000","last_clean":"2023-06-08T03:50:29.591398+0000","last_became_active":"2023-06-08T03:50:28.926614+0000","last_became_peered":"2023-06-08T03:50:28.926614+0000","last_unstale":"2023-06-08T03:50:29.591398+0000","last_undegraded":"2023-06-08T03:50:29.591398+0000","last_fullsized":"2023-06-08T03:50:29.591398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:45.910533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789240+0000","last_change":"2023-06-08T03:50:19.170601+0000","last_active":"2023-06-08T03:50:29.789240+0000","last_peered":"2023-06-08T03:50:29.789240+0000","last_clean":"2023-06-08T03:50:29.789240+0000","last_became_active":"2023-06-08T03:50:19.170344+0000","last_became_peered":"2023-06-08T03:50:19.170344+0000","last_unstale":"2023-06-08T03:50:29.789240+0000","last_undegraded":"2023-06-08T03:50:29.789240+0000","last_fullsized":"2023-06-08T03:50:29.789240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:35.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307871+0000","last_change":"2023-06-08T03:50:24.280451+0000","last_active":"2023-06-08T03:50:29.307871+0000","last_peered":"2023-06-08T03:50:29.307871+0000","last_clean":"2023-06-08T03:50:29.307871+0000","last_became_active":"2023-06-08T03:50:24.280118+0000","last_became_peered":"2023-06-08T03:50:24.280118+0000","last_unstale":"2023-06-08T03:50:29.307871+0000","last_undegraded":"2023-06-08T03:50:29.307871+0000","last_fullsized":"2023-06-08T03:50:29.307871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:18.146699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790813+0000","last_change":"2023-06-08T03:50:24.692660+0000","last_active":"2023-06-08T03:50:29.790813+0000","last_peered":"2023-06-08T03:50:29.790813+0000","last_clean":"2023-06-08T03:50:29.790813+0000","last_became_active":"2023-06-08T03:50:24.692373+0000","last_became_peered":"2023-06-08T03:50:24.692373+0000","last_unstale":"2023-06-08T03:50:29.790813+0000","last_undegraded":"2023-06-08T03:50:29.790813+0000","last_fullsized":"2023-06-08T03:50:29.790813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:14:48.949961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789804+0000","last_change":"2023-06-08T03:50:28.924634+0000","last_active":"2023-06-08T03:50:29.789804+0000","last_peered":"2023-06-08T03:50:29.789804+0000","last_clean":"2023-06-08T03:50:29.789804+0000","last_became_active":"2023-06-08T03:50:28.924227+0000","last_became_peered":"2023-06-08T03:50:28.924227+0000","last_unstale":"2023-06-08T03:50:29.789804+0000","last_undegraded":"2023-06-08T03:50:29.789804+0000","last_fullsized":"2023-06-08T03:50:29.789804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:24:05.389046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3,1],"acting":[7,4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789742+0000","last_change":"2023-06-08T03:50:19.805195+0000","last_active":"2023-06-08T03:50:29.789742+0000","last_peered":"2023-06-08T03:50:29.789742+0000","last_clean":"2023-06-08T03:50:29.789742+0000","last_became_active":"2023-06-08T03:50:19.804708+0000","last_became_peered":"2023-06-08T03:50:19.804708+0000","last_unstale":"2023-06-08T03:50:29.789742+0000","last_undegraded":"2023-06-08T03:50:29.789742+0000","last_fullsized":"2023-06-08T03:50:29.789742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:53:26.398016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591012+0000","last_change":"2023-06-08T03:50:24.281698+0000","last_active":"2023-06-08T03:50:29.591012+0000","last_peered":"2023-06-08T03:50:29.591012+0000","last_clean":"2023-06-08T03:50:29.591012+0000","last_became_active":"2023-06-08T03:50:24.281443+0000","last_became_peered":"2023-06-08T03:50:24.281443+0000","last_unstale":"2023-06-08T03:50:29.591012+0000","last_undegraded":"2023-06-08T03:50:29.591012+0000","last_fullsized":"2023-06-08T03:50:29.591012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:29.437443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309408+0000","last_change":"2023-06-08T03:50:24.693777+0000","last_active":"2023-06-08T03:50:29.309408+0000","last_peered":"2023-06-08T03:50:29.309408+0000","last_clean":"2023-06-08T03:50:29.309408+0000","last_became_active":"2023-06-08T03:50:24.693193+0000","last_became_peered":"2023-06-08T03:50:24.693193+0000","last_unstale":"2023-06-08T03:50:29.309408+0000","last_undegraded":"2023-06-08T03:50:29.309408+0000","last_fullsized":"2023-06-08T03:50:29.309408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591049+0000","last_change":"2023-06-08T03:50:28.926459+0000","last_active":"2023-06-08T03:50:29.591049+0000","last_peered":"2023-06-08T03:50:29.591049+0000","last_clean":"2023-06-08T03:50:29.591049+0000","last_became_active":"2023-06-08T03:50:28.926041+0000","last_became_peered":"2023-06-08T03:50:28.926041+0000","last_unstale":"2023-06-08T03:50:29.591049+0000","last_undegraded":"2023-06-08T03:50:29.591049+0000","last_fullsized":"2023-06-08T03:50:29.591049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:10.505175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,5],"acting":[0,7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:19.805012+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:19.804722+0000","last_became_peered":"2023-06-08T03:50:19.804722+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:43:32.916770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309261+0000","last_change":"2023-06-08T03:50:24.284390+0000","last_active":"2023-06-08T03:50:29.309261+0000","last_peered":"2023-06-08T03:50:29.309261+0000","last_clean":"2023-06-08T03:50:29.309261+0000","last_became_active":"2023-06-08T03:50:24.284129+0000","last_became_peered":"2023-06-08T03:50:24.284129+0000","last_unstale":"2023-06-08T03:50:29.309261+0000","last_undegraded":"2023-06-08T03:50:29.309261+0000","last_fullsized":"2023-06-08T03:50:29.309261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309328+0000","last_change":"2023-06-08T03:50:24.693789+0000","last_active":"2023-06-08T03:50:29.309328+0000","last_peered":"2023-06-08T03:50:29.309328+0000","last_clean":"2023-06-08T03:50:29.309328+0000","last_became_active":"2023-06-08T03:50:24.693205+0000","last_became_peered":"2023-06-08T03:50:24.693205+0000","last_unstale":"2023-06-08T03:50:29.309328+0000","last_undegraded":"2023-06-08T03:50:29.309328+0000","last_fullsized":"2023-06-08T03:50:29.309328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310499+0000","last_change":"2023-06-08T03:50:28.924218+0000","last_active":"2023-06-08T03:50:29.310499+0000","last_peered":"2023-06-08T03:50:29.310499+0000","last_clean":"2023-06-08T03:50:29.310499+0000","last_became_active":"2023-06-08T03:50:28.923473+0000","last_became_peered":"2023-06-08T03:50:28.923473+0000","last_unstale":"2023-06-08T03:50:29.310499+0000","last_undegraded":"2023-06-08T03:50:29.310499+0000","last_fullsized":"2023-06-08T03:50:29.310499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:11:10.428932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3,1],"acting":[4,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790922+0000","last_change":"2023-06-08T03:50:19.805107+0000","last_active":"2023-06-08T03:50:29.790922+0000","last_peered":"2023-06-08T03:50:29.790922+0000","last_clean":"2023-06-08T03:50:29.790922+0000","last_became_active":"2023-06-08T03:50:19.804754+0000","last_became_peered":"2023-06-08T03:50:19.804754+0000","last_unstale":"2023-06-08T03:50:29.790922+0000","last_undegraded":"2023-06-08T03:50:29.790922+0000","last_fullsized":"2023-06-08T03:50:29.790922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:13.049279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790682+0000","last_change":"2023-06-08T03:50:24.281311+0000","last_active":"2023-06-08T03:50:29.790682+0000","last_peered":"2023-06-08T03:50:29.790682+0000","last_clean":"2023-06-08T03:50:29.790682+0000","last_became_active":"2023-06-08T03:50:24.280858+0000","last_became_peered":"2023-06-08T03:50:24.280858+0000","last_unstale":"2023-06-08T03:50:29.790682+0000","last_undegraded":"2023-06-08T03:50:29.790682+0000","last_fullsized":"2023-06-08T03:50:29.790682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:01.113573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309301+0000","last_change":"2023-06-08T03:50:24.700010+0000","last_active":"2023-06-08T03:50:29.309301+0000","last_peered":"2023-06-08T03:50:29.309301+0000","last_clean":"2023-06-08T03:50:29.309301+0000","last_became_active":"2023-06-08T03:50:24.699801+0000","last_became_peered":"2023-06-08T03:50:24.699801+0000","last_unstale":"2023-06-08T03:50:29.309301+0000","last_undegraded":"2023-06-08T03:50:29.309301+0000","last_fullsized":"2023-06-08T03:50:29.309301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308391+0000","last_change":"2023-06-08T03:50:28.920073+0000","last_active":"2023-06-08T03:50:29.308391+0000","last_peered":"2023-06-08T03:50:29.308391+0000","last_clean":"2023-06-08T03:50:29.308391+0000","last_became_active":"2023-06-08T03:50:28.919601+0000","last_became_peered":"2023-06-08T03:50:28.919601+0000","last_unstale":"2023-06-08T03:50:29.308391+0000","last_undegraded":"2023-06-08T03:50:29.308391+0000","last_fullsized":"2023-06-08T03:50:29.308391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:38:02.965137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309419+0000","last_change":"2023-06-08T03:50:24.285734+0000","last_active":"2023-06-08T03:50:29.309419+0000","last_peered":"2023-06-08T03:50:29.309419+0000","last_clean":"2023-06-08T03:50:29.309419+0000","last_became_active":"2023-06-08T03:50:24.285421+0000","last_became_peered":"2023-06-08T03:50:24.285421+0000","last_unstale":"2023-06-08T03:50:29.309419+0000","last_undegraded":"2023-06-08T03:50:29.309419+0000","last_fullsized":"2023-06-08T03:50:29.309419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:13:35.050524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309586+0000","last_change":"2023-06-08T03:50:24.696432+0000","last_active":"2023-06-08T03:50:29.309586+0000","last_peered":"2023-06-08T03:50:29.309586+0000","last_clean":"2023-06-08T03:50:29.309586+0000","last_became_active":"2023-06-08T03:50:24.695650+0000","last_became_peered":"2023-06-08T03:50:24.695650+0000","last_unstale":"2023-06-08T03:50:29.309586+0000","last_undegraded":"2023-06-08T03:50:29.309586+0000","last_fullsized":"2023-06-08T03:50:29.309586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:33:18.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790496+0000","last_change":"2023-06-08T03:50:28.929940+0000","last_active":"2023-06-08T03:50:29.790496+0000","last_peered":"2023-06-08T03:50:29.790496+0000","last_clean":"2023-06-08T03:50:29.790496+0000","last_became_active":"2023-06-08T03:50:28.929641+0000","last_became_peered":"2023-06-08T03:50:28.929641+0000","last_unstale":"2023-06-08T03:50:29.790496+0000","last_undegraded":"2023-06-08T03:50:29.790496+0000","last_fullsized":"2023-06-08T03:50:29.790496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:40.205018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.285199+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.284917+0000","last_became_peered":"2023-06-08T03:50:24.284917+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790180+0000","last_change":"2023-06-08T03:50:24.680708+0000","last_active":"2023-06-08T03:50:29.790180+0000","last_peered":"2023-06-08T03:50:29.790180+0000","last_clean":"2023-06-08T03:50:29.790180+0000","last_became_active":"2023-06-08T03:50:24.680437+0000","last_became_peered":"2023-06-08T03:50:24.680437+0000","last_unstale":"2023-06-08T03:50:29.790180+0000","last_undegraded":"2023-06-08T03:50:29.790180+0000","last_fullsized":"2023-06-08T03:50:29.790180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:44.278509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789986+0000","last_change":"2023-06-08T03:50:28.922303+0000","last_active":"2023-06-08T03:50:29.789986+0000","last_peered":"2023-06-08T03:50:29.789986+0000","last_clean":"2023-06-08T03:50:29.789986+0000","last_became_active":"2023-06-08T03:50:28.921661+0000","last_became_peered":"2023-06-08T03:50:28.921661+0000","last_unstale":"2023-06-08T03:50:29.789986+0000","last_undegraded":"2023-06-08T03:50:29.789986+0000","last_fullsized":"2023-06-08T03:50:29.789986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:54.155089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,4],"acting":[2,7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790594+0000","last_change":"2023-06-08T03:50:24.287963+0000","last_active":"2023-06-08T03:50:29.790594+0000","last_peered":"2023-06-08T03:50:29.790594+0000","last_clean":"2023-06-08T03:50:29.790594+0000","last_became_active":"2023-06-08T03:50:24.287513+0000","last_became_peered":"2023-06-08T03:50:24.287513+0000","last_unstale":"2023-06-08T03:50:29.790594+0000","last_undegraded":"2023-06-08T03:50:29.790594+0000","last_fullsized":"2023-06-08T03:50:29.790594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:41:31.366674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789388+0000","last_change":"2023-06-08T03:50:24.694656+0000","last_active":"2023-06-08T03:50:29.789388+0000","last_peered":"2023-06-08T03:50:29.789388+0000","last_clean":"2023-06-08T03:50:29.789388+0000","last_became_active":"2023-06-08T03:50:24.694387+0000","last_became_peered":"2023-06-08T03:50:24.694387+0000","last_unstale":"2023-06-08T03:50:29.789388+0000","last_undegraded":"2023-06-08T03:50:29.789388+0000","last_fullsized":"2023-06-08T03:50:29.789388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:11:50.398160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789148+0000","last_change":"2023-06-08T03:50:28.926194+0000","last_active":"2023-06-08T03:50:29.789148+0000","last_peered":"2023-06-08T03:50:29.789148+0000","last_clean":"2023-06-08T03:50:29.789148+0000","last_became_active":"2023-06-08T03:50:28.925783+0000","last_became_peered":"2023-06-08T03:50:28.925783+0000","last_unstale":"2023-06-08T03:50:29.789148+0000","last_undegraded":"2023-06-08T03:50:29.789148+0000","last_fullsized":"2023-06-08T03:50:29.789148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:26.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5,7],"acting":[1,0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312099+0000","last_change":"2023-06-08T03:50:24.277779+0000","last_active":"2023-06-08T03:50:29.312099+0000","last_peered":"2023-06-08T03:50:29.312099+0000","last_clean":"2023-06-08T03:50:29.312099+0000","last_became_active":"2023-06-08T03:50:24.277493+0000","last_became_peered":"2023-06-08T03:50:24.277493+0000","last_unstale":"2023-06-08T03:50:29.312099+0000","last_undegraded":"2023-06-08T03:50:29.312099+0000","last_fullsized":"2023-06-08T03:50:29.312099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789292+0000","last_change":"2023-06-08T03:50:24.696150+0000","last_active":"2023-06-08T03:50:29.789292+0000","last_peered":"2023-06-08T03:50:29.789292+0000","last_clean":"2023-06-08T03:50:29.789292+0000","last_became_active":"2023-06-08T03:50:24.695605+0000","last_became_peered":"2023-06-08T03:50:24.695605+0000","last_unstale":"2023-06-08T03:50:29.789292+0000","last_undegraded":"2023-06-08T03:50:29.789292+0000","last_fullsized":"2023-06-08T03:50:29.789292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:28:06.091963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308122+0000","last_change":"2023-06-08T03:50:28.924072+0000","last_active":"2023-06-08T03:50:29.308122+0000","last_peered":"2023-06-08T03:50:29.308122+0000","last_clean":"2023-06-08T03:50:29.308122+0000","last_became_active":"2023-06-08T03:50:28.923155+0000","last_became_peered":"2023-06-08T03:50:28.923155+0000","last_unstale":"2023-06-08T03:50:29.308122+0000","last_undegraded":"2023-06-08T03:50:29.308122+0000","last_fullsized":"2023-06-08T03:50:29.308122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:00:25.670438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1,0],"acting":[4,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312115+0000","last_change":"2023-06-08T03:50:24.285764+0000","last_active":"2023-06-08T03:50:29.312115+0000","last_peered":"2023-06-08T03:50:29.312115+0000","last_clean":"2023-06-08T03:50:29.312115+0000","last_became_active":"2023-06-08T03:50:24.285477+0000","last_became_peered":"2023-06-08T03:50:24.285477+0000","last_unstale":"2023-06-08T03:50:29.312115+0000","last_undegraded":"2023-06-08T03:50:29.312115+0000","last_fullsized":"2023-06-08T03:50:29.312115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:41.022058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.788785+0000","last_change":"2023-06-08T03:50:24.679506+0000","last_active":"2023-06-08T03:50:29.788785+0000","last_peered":"2023-06-08T03:50:29.788785+0000","last_clean":"2023-06-08T03:50:29.788785+0000","last_became_active":"2023-06-08T03:50:24.679246+0000","last_became_peered":"2023-06-08T03:50:24.679246+0000","last_unstale":"2023-06-08T03:50:29.788785+0000","last_undegraded":"2023-06-08T03:50:29.788785+0000","last_fullsized":"2023-06-08T03:50:29.788785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:53:20.880498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789376+0000","last_change":"2023-06-08T03:50:24.690393+0000","last_active":"2023-06-08T03:50:29.789376+0000","last_peered":"2023-06-08T03:50:29.789376+0000","last_clean":"2023-06-08T03:50:29.789376+0000","last_became_active":"2023-06-08T03:50:24.690144+0000","last_became_peered":"2023-06-08T03:50:24.690144+0000","last_unstale":"2023-06-08T03:50:29.789376+0000","last_undegraded":"2023-06-08T03:50:29.789376+0000","last_fullsized":"2023-06-08T03:50:29.789376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309500+0000","last_change":"2023-06-08T03:50:24.693278+0000","last_active":"2023-06-08T03:50:29.309500+0000","last_peered":"2023-06-08T03:50:29.309500+0000","last_clean":"2023-06-08T03:50:29.309500+0000","last_became_active":"2023-06-08T03:50:24.692978+0000","last_became_peered":"2023-06-08T03:50:24.692978+0000","last_unstale":"2023-06-08T03:50:29.309500+0000","last_undegraded":"2023-06-08T03:50:29.309500+0000","last_fullsized":"2023-06-08T03:50:29.309500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591028+0000","last_change":"2023-06-08T03:50:24.700428+0000","last_active":"2023-06-08T03:50:29.591028+0000","last_peered":"2023-06-08T03:50:29.591028+0000","last_clean":"2023-06-08T03:50:29.591028+0000","last_became_active":"2023-06-08T03:50:24.700158+0000","last_became_peered":"2023-06-08T03:50:24.700158+0000","last_unstale":"2023-06-08T03:50:29.591028+0000","last_undegraded":"2023-06-08T03:50:29.591028+0000","last_fullsized":"2023-06-08T03:50:29.591028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310861+0000","last_change":"2023-06-08T03:50:24.690477+0000","last_active":"2023-06-08T03:50:29.310861+0000","last_peered":"2023-06-08T03:50:29.310861+0000","last_clean":"2023-06-08T03:50:29.310861+0000","last_became_active":"2023-06-08T03:50:24.690125+0000","last_became_peered":"2023-06-08T03:50:24.690125+0000","last_unstale":"2023-06-08T03:50:29.310861+0000","last_undegraded":"2023-06-08T03:50:29.310861+0000","last_fullsized":"2023-06-08T03:50:29.310861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:40.966384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311958+0000","last_change":"2023-06-08T03:50:24.682936+0000","last_active":"2023-06-08T03:50:29.311958+0000","last_peered":"2023-06-08T03:50:29.311958+0000","last_clean":"2023-06-08T03:50:29.311958+0000","last_became_active":"2023-06-08T03:50:24.681545+0000","last_became_peered":"2023-06-08T03:50:24.681545+0000","last_unstale":"2023-06-08T03:50:29.311958+0000","last_undegraded":"2023-06-08T03:50:29.311958+0000","last_fullsized":"2023-06-08T03:50:29.311958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4451,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":270336,"data_stored":109499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":110083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":109555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":113062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":880640,"data_stored":703534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":700359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-08T03:50:43.409 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-08T03:50:43.409 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-08T03:50:43.409 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-08T03:50:43.410 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-08T03:50:43.410 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-08T03:50:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.558+0000 7f25b8076700 1 Processor -- start 2023-06-08T03:50:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.558+0000 7f25b8076700 1 -- start start 2023-06-08T03:50:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.558+0000 7f25b8076700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b8076700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25b0160080 0x7f25b00cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b8076700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00cea60 con 0x7f25b00b1650 2023-06-08T03:50:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b8076700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00cf6e0 con 0x7f25b0160080 2023-06-08T03:50:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b8076700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ced40 con 0x7f25b006b0d0 2023-06-08T03:50:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b6613700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f25b0160080 0x7f25b00cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b5611700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f25b00b1650 0x7f25b015f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58370/0 (socket says 172.21.15.79:58370) 2023-06-08T03:50:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b5611700 1 -- 172.21.15.79:0/4282121445 learned_addr learned my addr 172.21.15.79:0/4282121445 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b6613700 1 -- 172.21.15.79:0/4282121445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 msgr2=0x7f25b00ce460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-08T03:50:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b6613700 1 --2- 172.21.15.79:0/4282121445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00ce460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b6613700 1 -- 172.21.15.79:0/4282121445 >> v1:172.21.15.79:6789/0 conn(0x7f25b00b1650 legacy=0x7f25b015f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b6613700 1 -- 172.21.15.79:0/4282121445 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00cf020 con 0x7f25b0160080 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b4e10700 1 -- 172.21.15.79:0/4282121445 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042171810 0 0) 0x7f25b00cea60 con 0x7f25b00b1650 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b6613700 1 --2- 172.21.15.79:0/4282121445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b0160080 0x7f25b00cfcd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009d90 tx=0x7f25a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d2eeac011d32b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b4e10700 1 -- 172.21.15.79:0/4282121445 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800e070 con 0x7f25b0160080 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b4e10700 1 -- 172.21.15.79:0/4282121445 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a800bbd0 con 0x7f25b0160080 2023-06-08T03:50:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.559+0000 7f25b4e10700 1 -- 172.21.15.79:0/4282121445 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8012430 con 0x7f25b0160080 2023-06-08T03:50:43.567 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 -- 172.21.15.79:0/4282121445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b0160080 msgr2=0x7f25b00cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 --2- 172.21.15.79:0/4282121445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b0160080 0x7f25b00cfcd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009d90 tx=0x7f25a800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:43.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 -- 172.21.15.79:0/4282121445 shutdown_connections 2023-06-08T03:50:43.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 --2- 172.21.15.79:0/4282121445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b0160080 0x7f25b00cfcd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 --2- 172.21.15.79:0/4282121445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.568 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 -- 172.21.15.79:0/4282121445 >> 172.21.15.79:0/4282121445 conn(0x7f25b006a670 msgr2=0x7f25b0099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 -- 172.21.15.79:0/4282121445 shutdown_connections 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.560+0000 7f25b8076700 1 -- 172.21.15.79:0/4282121445 wait complete. 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.561+0000 7f25b8076700 1 Processor -- start 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.561+0000 7f25b8076700 1 -- start start 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 0x7f25b00c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b0160080 0x7f25b01f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.569 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00af6e0 con 0x7f25b00b1650 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b015a9d0 con 0x7f25b0160080 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ae0d0 con 0x7f25b006b0d0 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 0x7f25b00c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 0x7f25b00c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34900/0 (socket says 172.21.15.79:34900) 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 -- 172.21.15.79:0/2373108945 learned_addr learned my addr 172.21.15.79:0/2373108945 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5e12700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b6613700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b0160080 0x7f25b01f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.570 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 -- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 msgr2=0x7f25b00af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 -- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b0160080 msgr2=0x7f25b01f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b0160080 0x7f25b01f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 -- 172.21.15.79:0/2373108945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8009a70 con 0x7f25b00b1650 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b5611700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 0x7f25b00c5ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0011e0 tx=0x7f25ac00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac004c80 con 0x7f25b00b1650 2023-06-08T03:50:43.571 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac004e00 con 0x7f25b00b1650 2023-06-08T03:50:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b00ae410 con 0x7f25b00b1650 2023-06-08T03:50:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.562+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac007310 con 0x7f25b00b1650 2023-06-08T03:50:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.563+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b00aec50 con 0x7f25b00b1650 2023-06-08T03:50:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.564+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b00c4330 con 0x7f25b00b1650 2023-06-08T03:50:43.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.564+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f25ac007470 con 0x7f25b00b1650 2023-06-08T03:50:43.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.564+0000 7f25a67fc700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f259406ace0 0x7f259406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.564+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f25ac089480 con 0x7f25b00b1650 2023-06-08T03:50:43.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.565+0000 7f25b5e12700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f259406ace0 0x7f259406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.565+0000 7f25b5e12700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f259406ace0 0x7f259406d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009910 tx=0x7f25a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.569+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f25ac0542d0 con 0x7f25b00b1650 2023-06-08T03:50:43.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.814+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f25b0073e90 con 0x7f25b00b1650 2023-06-08T03:50:43.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.817+0000 7f25a67fc700 1 -- 172.21.15.79:0/2373108945 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170849 (secure 0 0 0) 0x7f25ac0578f0 con 0x7f25b00b1650 2023-06-08T03:50:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:43.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f259406ace0 msgr2=0x7f259406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f259406ace0 0x7f259406d190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009910 tx=0x7f25a0008040 comp rx=0 tx=0).stop 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 msgr2=0x7f25b00c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 0x7f25b00c5ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0011e0 tx=0x7f25ac00cae0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 shutdown_connections 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25b0160080 0x7f25b01f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f259406ace0 0x7f259406d190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f25b006b0d0 0x7f25b00af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 --2- 172.21.15.79:0/2373108945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f25b00b1650 0x7f25b00c5ea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 >> 172.21.15.79:0/2373108945 conn(0x7f25b006a670 msgr2=0x7f25b00d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:43.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 shutdown_connections 2023-06-08T03:50:43.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.819+0000 7f25b8076700 1 -- 172.21.15.79:0/2373108945 wait complete. 2023-06-08T03:50:43.835 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":5,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2537078349},{"type":"v1","addr":"172.21.15.156:6833","nonce":2537078349}]},"active_addr":"172.21.15.156:6833/2537078349","active_change":"2023-06-08T03:50:11.860612+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":1192156543}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":1208004780}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":1319516727}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.156:0","nonce":1441532197}]}]} 2023-06-08T03:50:43.839 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-08T03:50:43.840 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-08T03:50:43.840 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:43.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 Processor -- start 2023-06-08T03:50:43.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 -- start start 2023-06-08T03:50:43.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 0x7fbc900b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbc9015aa10 0x7fbc900ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900b0750 con 0x7fbc900ce930 2023-06-08T03:50:43.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc900ceeb0 con 0x7fbc9015aa10 2023-06-08T03:50:43.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.958+0000 7fbc9500f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900b09a0 con 0x7fbc9015a640 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8f59e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbc900ce930 0x7fbc900af580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58386/0 (socket says 172.21.15.79:58386) 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8f59e700 1 -- 172.21.15.79:0/2035442317 learned_addr learned my addr 172.21.15.79:0/2035442317 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2035442317 >> v2:172.21.15.156:3300/0 conn(0x7fbc9015aa10 0x7fbc900ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8ed9d700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 0x7fbc900b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63806971 0 0) 0x7fbc900b0750 con 0x7fbc900ce930 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc780035b0 con 0x7fbc900ce930 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480038714 0 0) 0x7fbc780035b0 con 0x7fbc900ce930 2023-06-08T03:50:43.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 msgr2=0x7fbc900b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 0x7fbc900b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 >> v2:172.21.15.156:3300/0 conn(0x7fbc9015aa10 msgr2=0x7fbc900ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 --2- 172.21.15.79:0/2035442317 >> v2:172.21.15.156:3300/0 conn(0x7fbc9015aa10 0x7fbc900ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900b0c80 con 0x7fbc900ce930 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fbc880019f0 con 0x7fbc900ce930 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2035442317 >> v2:172.21.15.156:3300/0 conn(0x7fbc9015aa10 0x7fbc900ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:43.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 >> v1:172.21.15.79:6789/0 conn(0x7fbc900ce930 legacy=0x7fbc900af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 0x7fbc78003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc78003ff0 0x7fbc78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78004d90 0x7fbc78005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900ceeb0 con 0x7fbc78003ff0 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900b09a0 con 0x7fbc78004d90 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7800ba10 con 0x7fbc9015a640 2023-06-08T03:50:43.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc78003ff0 0x7fbc78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8f59e700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78004d90 0x7fbc78005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8ed9d700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 0x7fbc78003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 -- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 msgr2=0x7fbc78003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc9015a640 0x7fbc78003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 -- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78004d90 msgr2=0x7fbc78005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78004d90 0x7fbc78005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.959+0000 7fbc8e59c700 1 -- 172.21.15.79:0/2035442317 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc780035b0 con 0x7fbc78003ff0 2023-06-08T03:50:43.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc78003ff0 0x7fbc78004850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009d60 tx=0x7fbc78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f73910d5aa28a60 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8000e9a0 con 0x7fbc78003ff0 2023-06-08T03:50:43.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8000eb00 con 0x7fbc78003ff0 2023-06-08T03:50:43.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc8dd9b700 1 -- 172.21.15.79:0/2035442317 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8000eeb0 con 0x7fbc78003ff0 2023-06-08T03:50:43.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc9500f700 1 -- 172.21.15.79:0/2035442317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc78003ff0 msgr2=0x7fbc78004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2035442317 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc78003ff0 0x7fbc78004850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009d60 tx=0x7fbc78005880 comp rx=0 tx=0).stop 2023-06-08T03:50:43.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc8f59e700 1 -- 172.21.15.79:0/2035442317 reap_dead start 2023-06-08T03:50:43.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc9500f700 1 -- 172.21.15.79:0/2035442317 shutdown_connections 2023-06-08T03:50:43.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc9500f700 1 -- 172.21.15.79:0/2035442317 >> 172.21.15.79:0/2035442317 conn(0x7fbc9006a670 msgr2=0x7fbc90099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:43.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc9500f700 1 -- 172.21.15.79:0/2035442317 shutdown_connections 2023-06-08T03:50:43.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.960+0000 7fbc9500f700 1 -- 172.21.15.79:0/2035442317 wait complete. 2023-06-08T03:50:43.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.962+0000 7fbc9500f700 1 Processor -- start 2023-06-08T03:50:43.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.962+0000 7fbc9500f700 1 -- start start 2023-06-08T03:50:43.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.962+0000 7fbc9500f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78003ff0 0x7fbc90153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.962+0000 7fbc9500f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc90152a50 0x7fbc901596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc9500f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 0x7fbc901f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc9500f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901f35a0 con 0x7fbc900b4c30 2023-06-08T03:50:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc9500f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90152f10 con 0x7fbc78003ff0 2023-06-08T03:50:43.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc9500f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901531f0 con 0x7fbc90152a50 2023-06-08T03:50:43.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 0x7fbc901f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 0x7fbc901f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34920/0 (socket says 172.21.15.79:34920) 2023-06-08T03:50:43.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 -- 172.21.15.79:0/2652097890 learned_addr learned my addr 172.21.15.79:0/2652097890 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:43.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8ed9d700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78003ff0 0x7fbc90153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc90152a50 0x7fbc901596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 -- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc90152a50 msgr2=0x7fbc901596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc90152a50 0x7fbc901596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 -- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78003ff0 msgr2=0x7fbc90153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78003ff0 0x7fbc90153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 -- 172.21.15.79:0/2652097890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc901534d0 con 0x7fbc900b4c30 2023-06-08T03:50:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8e59c700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc90152a50 0x7fbc901596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc8f59e700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 0x7fbc901f3a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc900672e0 tx=0x7fbc88007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800ec00 con 0x7fbc900b4c30 2023-06-08T03:50:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800ed80 con 0x7fbc900b4c30 2023-06-08T03:50:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc901537b0 con 0x7fbc900b4c30 2023-06-08T03:50:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88013660 con 0x7fbc900b4c30 2023-06-08T03:50:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.963+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc90159c30 con 0x7fbc900b4c30 2023-06-08T03:50:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.964+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc90159e20 con 0x7fbc900b4c30 2023-06-08T03:50:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.965+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc880137c0 con 0x7fbc900b4c30 2023-06-08T03:50:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.965+0000 7fbc777fe700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbc7c06ace0 0x7fbc7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.965+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbc88089cc0 con 0x7fbc900b4c30 2023-06-08T03:50:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.965+0000 7fbc8ed9d700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbc7c06ace0 0x7fbc7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.966+0000 7fbc8ed9d700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbc7c06ace0 0x7fbc7c06d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009a80 tx=0x7fbc84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:43.969+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbc88054a80 con 0x7fbc900b4c30 2023-06-08T03:50:44.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.159+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc90159e20 con 0x7fbc900b4c30 2023-06-08T03:50:44.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.160+0000 7fbc777fe700 1 -- 172.21.15.79:0/2652097890 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fbc880580a0 con 0x7fbc900b4c30 2023-06-08T03:50:44.164 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:44.164 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:44.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbc7c06ace0 msgr2=0x7fbc7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbc7c06ace0 0x7fbc7c06d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009a80 tx=0x7fbc84008040 comp rx=0 tx=0).stop 2023-06-08T03:50:44.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 msgr2=0x7fbc901f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 0x7fbc901f3a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc900672e0 tx=0x7fbc88007c90 comp rx=0 tx=0).stop 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 shutdown_connections 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc900b4c30 0x7fbc901f3a60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbc78003ff0 0x7fbc90153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbc7c06ace0 0x7fbc7c06d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 --2- 172.21.15.79:0/2652097890 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbc90152a50 0x7fbc901596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 >> 172.21.15.79:0/2652097890 conn(0x7fbc9006a670 msgr2=0x7fbc90072800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 shutdown_connections 2023-06-08T03:50:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.162+0000 7fbc9500f700 1 -- 172.21.15.79:0/2652097890 wait complete. 2023-06-08T03:50:44.179 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-08T03:50:44.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:44.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.295+0000 7fd1c4fb2700 1 Processor -- start 2023-06-08T03:50:44.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.295+0000 7fd1c4fb2700 1 -- start start 2023-06-08T03:50:44.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.296+0000 7fd1c4fb2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd1c006ae80 0x7fd1c015c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.296+0000 7fd1c4fb2700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00b20f0 con 0x7fd1c00b1d20 2023-06-08T03:50:44.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.296+0000 7fd1c4fb2700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c006b310 con 0x7fd1c00d2420 2023-06-08T03:50:44.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.296+0000 7fd1c4fb2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c006b5f0 con 0x7fd1c006ae80 2023-06-08T03:50:44.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1be59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd1c00b1d20 0x7fd1c00ccef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58402/0 (socket says 172.21.15.79:58402) 2023-06-08T03:50:44.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1be59c700 1 -- 172.21.15.79:0/3960515438 learned_addr learned my addr 172.21.15.79:0/3960515438 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:44.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bdd9b700 1 --2- 172.21.15.79:0/3960515438 >> v2:172.21.15.156:3300/0 conn(0x7fd1c006ae80 0x7fd1c015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498458460 0 0) 0x7fd1c00b20f0 con 0x7fd1c00b1d20 2023-06-08T03:50:44.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a80035b0 con 0x7fd1c00b1d20 2023-06-08T03:50:44.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709115329 0 0) 0x7fd1a80035b0 con 0x7fd1c00b1d20 2023-06-08T03:50:44.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 >> v2:172.21.15.156:3300/0 conn(0x7fd1c006ae80 msgr2=0x7fd1c015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 --2- 172.21.15.79:0/3960515438 >> v2:172.21.15.156:3300/0 conn(0x7fd1c006ae80 0x7fd1c015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 >> v1:172.21.15.156:6789/0 conn(0x7fd1c00d2420 legacy=0x7fd1c006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c006b8d0 con 0x7fd1c00b1d20 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fd1b4001670 con 0x7fd1c00b1d20 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 >> v1:172.21.15.79:6789/0 conn(0x7fd1c00b1d20 legacy=0x7fd1c00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1a8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a8004850 0x7fd1a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1a8005200 0x7fd1a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c006b5f0 con 0x7fd1a8003a70 2023-06-08T03:50:44.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a800c0e0 con 0x7fd1a8004850 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a800c2a0 con 0x7fd1a8005200 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1be59c700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1a8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bdd9b700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a8004850 0x7fd1a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.297+0000 7fd1bed9d700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1a8005200 0x7fd1a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1be59c700 1 -- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1a8005200 msgr2=0x7fd1a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1be59c700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1a8005200 0x7fd1a800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1be59c700 1 -- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a8004850 msgr2=0x7fd1a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1be59c700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1a8004850 0x7fd1a8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1bed9d700 1 -- 172.21.15.79:0/3960515438 reap_dead start 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1be59c700 1 -- 172.21.15.79:0/3960515438 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80035b0 con 0x7fd1a8003a70 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1be59c700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1a8003e80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c240 tx=0x7fd1a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7faeaae74dd59e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007c10 con 0x7fd1a8003a70 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400cde0 con 0x7fd1a8003a70 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1bd59a700 1 -- 172.21.15.79:0/3960515438 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4005ba0 con 0x7fd1a8003a70 2023-06-08T03:50:44.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/3960515438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 msgr2=0x7fd1a8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1a8003e80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c240 tx=0x7fd1a8005880 comp rx=0 tx=0).stop 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/3960515438 shutdown_connections 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/3960515438 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1a8003e80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.298+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/3960515438 >> 172.21.15.79:0/3960515438 conn(0x7fd1c009bbb0 msgr2=0x7fd1c00d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.299+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/3960515438 shutdown_connections 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.299+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/3960515438 wait complete. 2023-06-08T03:50:44.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.300+0000 7fd1c4fb2700 1 Processor -- start 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.300+0000 7fd1c4fb2700 1 -- start start 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1c009d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c01e25f0 0x7fd1c01f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00c8060 0x7fd1c00c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00c8a40 con 0x7fd1a8003a70 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c01e2d90 con 0x7fd1c01e25f0 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0147c70 con 0x7fd1c00c8060 2023-06-08T03:50:44.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1c009d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1c009d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34938/0 (socket says 172.21.15.79:34938) 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 -- 172.21.15.79:0/594283032 learned_addr learned my addr 172.21.15.79:0/594283032 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1bdd9b700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c01e25f0 0x7fd1c01f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 -- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00c8060 msgr2=0x7fd1c00c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1bed9d700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00c8060 0x7fd1c00c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00c8060 0x7fd1c00c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 -- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c01e25f0 msgr2=0x7fd1c01f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c01e25f0 0x7fd1c01f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 -- 172.21.15.79:0/594283032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0147f50 con 0x7fd1a8003a70 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1bdd9b700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c01e25f0 0x7fd1c01f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1be59c700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1c009d1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005830 tx=0x7fd1b4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b40070d0 con 0x7fd1a8003a70 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4007250 con 0x7fd1a8003a70 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c0148230 con 0x7fd1a8003a70 2023-06-08T03:50:44.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.301+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007600 con 0x7fd1a8003a70 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.302+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c01e2a60 con 0x7fd1a8003a70 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.303+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c008cc30 con 0x7fd1a8003a70 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.303+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1b4027450 con 0x7fd1a8003a70 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.303+0000 7fd1aeffd700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd1a006ace0 0x7fd1a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.303+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd1b408acc0 con 0x7fd1a8003a70 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.304+0000 7fd1bdd9b700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd1a006ace0 0x7fd1a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.304+0000 7fd1bdd9b700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd1a006ace0 0x7fd1a006d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd1c015b7c0 tx=0x7fd1b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.308+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd1b4055a90 con 0x7fd1a8003a70 2023-06-08T03:50:44.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.493+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c0073e90 con 0x7fd1a8003a70 2023-06-08T03:50:44.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.495+0000 7fd1aeffd700 1 -- 172.21.15.79:0/594283032 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fd1b40590b0 con 0x7fd1a8003a70 2023-06-08T03:50:44.499 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:44.499 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:44.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd1a006ace0 msgr2=0x7fd1a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd1a006ace0 0x7fd1a006d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd1c015b7c0 tx=0x7fd1b0008040 comp rx=0 tx=0).stop 2023-06-08T03:50:44.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 msgr2=0x7fd1c009d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1c009d1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005830 tx=0x7fd1b4001a00 comp rx=0 tx=0).stop 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 shutdown_connections 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd1a006ace0 0x7fd1a006d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1c00c8060 0x7fd1c00c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd1a8003a70 0x7fd1c009d1a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 --2- 172.21.15.79:0/594283032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1c01e25f0 0x7fd1c01f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 >> 172.21.15.79:0/594283032 conn(0x7fd1c009bbb0 msgr2=0x7fd1c00b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 shutdown_connections 2023-06-08T03:50:44.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.497+0000 7fd1c4fb2700 1 -- 172.21.15.79:0/594283032 wait complete. 2023-06-08T03:50:44.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-08T03:50:44.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-08T03:50:44.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-08T03:50:44.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-08T03:50:44.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-06-08T03:50:44.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-06-08T03:50:44.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-06-08T03:50:44.517 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-06-08T03:50:44.716 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.710+0000 7fc96697c700 1 Processor -- start 2023-06-08T03:50:44.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.710+0000 7fc96697c700 1 -- start start 2023-06-08T03:50:44.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.711+0000 7fc96697c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9600cfbe0 0x7fc9600cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.711+0000 7fc96697c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9600cdf00 con 0x7fc96006cb60 2023-06-08T03:50:44.717 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.711+0000 7fc96697c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96006bc80 con 0x7fc96006c6f0 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.711+0000 7fc96697c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600d04f0 con 0x7fc9600cfbe0 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc964f19700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc96006cb60 0x7fc96006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58424/0 (socket says 172.21.15.79:58424) 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc964f19700 1 -- 172.21.15.79:0/3547001489 learned_addr learned my addr 172.21.15.79:0/3547001489 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc95ffff700 1 --2- 172.21.15.79:0/3547001489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9600cfbe0 0x7fc9600cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc95ffff700 1 -- 172.21.15.79:0/3547001489 >> v1:172.21.15.156:6789/0 conn(0x7fc96006c6f0 legacy=0x7fc96006ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc95ffff700 1 -- 172.21.15.79:0/3547001489 >> v1:172.21.15.79:6789/0 conn(0x7fc96006cb60 legacy=0x7fc96006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc95ffff700 1 -- 172.21.15.79:0/3547001489 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600cd2f0 con 0x7fc9600cfbe0 2023-06-08T03:50:44.718 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc95effd700 1 -- 172.21.15.79:0/3547001489 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179100357 0 0) 0x7fc9600cdf00 con 0x7fc96006cb60 2023-06-08T03:50:44.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.712+0000 7fc95ffff700 1 --2- 172.21.15.79:0/3547001489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9600cfbe0 0x7fc9600cffb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc958009dc0 tx=0x7fc95800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb3b82f83376193 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.713+0000 7fc95effd700 1 -- 172.21.15.79:0/3547001489 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800e070 con 0x7fc9600cfbe0 2023-06-08T03:50:44.719 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.713+0000 7fc95effd700 1 -- 172.21.15.79:0/3547001489 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800bb90 con 0x7fc9600cfbe0 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.713+0000 7fc95effd700 1 -- 172.21.15.79:0/3547001489 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958003a20 con 0x7fc9600cfbe0 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.714+0000 7fc96697c700 1 -- 172.21.15.79:0/3547001489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9600cfbe0 msgr2=0x7fc9600cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.714+0000 7fc96697c700 1 --2- 172.21.15.79:0/3547001489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9600cfbe0 0x7fc9600cffb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc958009dc0 tx=0x7fc95800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.714+0000 7fc96697c700 1 -- 172.21.15.79:0/3547001489 shutdown_connections 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.714+0000 7fc96697c700 1 --2- 172.21.15.79:0/3547001489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9600cfbe0 0x7fc9600cffb0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc958009dc0 tx=0x7fc95800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.714+0000 7fc96697c700 1 -- 172.21.15.79:0/3547001489 >> 172.21.15.79:0/3547001489 conn(0x7fc960069d50 msgr2=0x7fc96006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.720 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.715+0000 7fc96697c700 1 -- 172.21.15.79:0/3547001489 shutdown_connections 2023-06-08T03:50:44.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.715+0000 7fc96697c700 1 -- 172.21.15.79:0/3547001489 wait complete. 2023-06-08T03:50:44.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.717+0000 7fc96697c700 1 Processor -- start 2023-06-08T03:50:44.721 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.718+0000 7fc96697c700 1 -- start start 2023-06-08T03:50:44.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.718+0000 7f42d8a14700 1 Processor -- start 2023-06-08T03:50:44.722 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc96697c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc96006c6f0 0x7fc960156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc96697c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc96006cb60 0x7fc9600c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc96697c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9600b0990 0x7fc9600b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc96697c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600b1350 con 0x7fc96006c6f0 2023-06-08T03:50:44.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc96697c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600b6390 con 0x7fc9600b0990 2023-06-08T03:50:44.724 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc96697c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600d4ce0 con 0x7fc96006cb60 2023-06-08T03:50:44.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc95f7fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc96006cb60 0x7fc9600c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc95f7fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc96006cb60 0x7fc9600c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49274/0 (socket says 172.21.15.79:49274) 2023-06-08T03:50:44.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc95f7fe700 1 -- 172.21.15.79:0/1703124236 learned_addr learned my addr 172.21.15.79:0/1703124236 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.725 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7fc964f19700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9600b0990 0x7fc9600b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc964f19700 1 -- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc96006cb60 msgr2=0x7fc9600c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc964f19700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc96006cb60 0x7fc9600c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc964f19700 1 -- 172.21.15.79:0/1703124236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc96006c6f0 msgr2=0x7fc960156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc964f19700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc96006c6f0 0x7fc960156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.726 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc964f19700 1 -- 172.21.15.79:0/1703124236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc958009a70 con 0x7fc9600b0990 2023-06-08T03:50:44.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.718+0000 7f42d8a14700 1 -- start start 2023-06-08T03:50:44.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d8a14700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f42d40ca160 0x7f42d40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d8a14700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f42d40af8d0 0x7f42d40713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d8a14700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d40cda10 con 0x7f42d4071920 2023-06-08T03:50:44.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d8a14700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d40ca530 con 0x7f42d40af8d0 2023-06-08T03:50:44.727 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d8a14700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d40cb640 con 0x7f42d40ca160 2023-06-08T03:50:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d259c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f42d40ca160 0x7f42d40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d259c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f42d40ca160 0x7f42d40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:57852/0 (socket says 172.21.15.79:57852) 2023-06-08T03:50:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f42d259c700 1 -- 172.21.15.79:0/200769170 learned_addr learned my addr 172.21.15.79:0/200769170 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.718+0000 7f38d283c700 1 Processor -- start 2023-06-08T03:50:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.719+0000 7f38d283c700 1 -- start start 2023-06-08T03:50:44.728 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38d283c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38d283c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f38cc0ccfe0 0x7f38cc0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38d283c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc0cc9c0 con 0x7f38cc0cb890 2023-06-08T03:50:44.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38d283c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0ccbc0 con 0x7f38cc0ccfe0 2023-06-08T03:50:44.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38d283c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc1609e0 con 0x7f38cc0c14b0 2023-06-08T03:50:44.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc964f19700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9600b0990 0x7fc9600b0e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc960072530 tx=0x7fc95000c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38cb7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f38cc0cb890 0x7f38cc0c1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60598/0 (socket says 172.21.15.79:60598) 2023-06-08T03:50:44.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7f38cb7fe700 1 -- 172.21.15.79:0/786404955 learned_addr learned my addr 172.21.15.79:0/786404955 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:44.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950004c80 con 0x7fc9600b0990 2023-06-08T03:50:44.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95000ce90 con 0x7fc9600b0990 2023-06-08T03:50:44.730 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600d4fc0 con 0x7fc9600b0990 2023-06-08T03:50:44.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f42d1d9b700 1 --2- 172.21.15.79:0/200769170 >> v2:172.21.15.79:3300/0 conn(0x7f42d40af8d0 0x7f42d40713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38d0dd9700 1 --2- 172.21.15.79:0/786404955 >> v2:172.21.15.79:3300/0 conn(0x7f38cc0ccfe0 0x7f38cc0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.731 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38cbfff700 1 --2- 172.21.15.79:0/786404955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38caffd700 1 -- 172.21.15.79:0/786404955 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953300881 0 0) 0x7f38cc0cc9c0 con 0x7f38cc0cb890 2023-06-08T03:50:44.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38caffd700 1 -- 172.21.15.79:0/786404955 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b40035b0 con 0x7f38cc0cb890 2023-06-08T03:50:44.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38d0dd9700 1 -- 172.21.15.79:0/786404955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 msgr2=0x7f38cc0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38d0dd9700 1 --2- 172.21.15.79:0/786404955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc0ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38d0dd9700 1 -- 172.21.15.79:0/786404955 >> v1:172.21.15.156:6789/0 conn(0x7f38cc0cb890 legacy=0x7f38cc0c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.732 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f38d0dd9700 1 -- 172.21.15.79:0/786404955 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0ae990 con 0x7f38cc0ccfe0 2023-06-08T03:50:44.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500070d0 con 0x7fc9600b0990 2023-06-08T03:50:44.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.720+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9600c4400 con 0x7fc9600b0990 2023-06-08T03:50:44.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f42d159a700 1 -- 172.21.15.79:0/200769170 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347972147 0 0) 0x7f42d40cda10 con 0x7f42d4071920 2023-06-08T03:50:44.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f42d259c700 1 -- 172.21.15.79:0/200769170 >> v2:172.21.15.79:3300/0 conn(0x7f42d40af8d0 msgr2=0x7f42d40713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f42d259c700 1 --2- 172.21.15.79:0/200769170 >> v2:172.21.15.79:3300/0 conn(0x7f42d40af8d0 0x7f42d40713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.733 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f42d259c700 1 -- 172.21.15.79:0/200769170 >> v1:172.21.15.156:6789/0 conn(0x7f42d4071920 legacy=0x7f42d40c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.721+0000 7f42d259c700 1 -- 172.21.15.79:0/200769170 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d4071d20 con 0x7f42d40ca160 2023-06-08T03:50:44.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f38d0dd9700 1 --2- 172.21.15.79:0/786404955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 0x7f38cc0cd460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009d90 tx=0x7f38c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6473bc6442de153 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.734 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9600cd8b0 con 0x7fc9600b0990 2023-06-08T03:50:44.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f42d259c700 1 --2- 172.21.15.79:0/200769170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d40ca160 0x7f42d40cde80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42d40cf4f0 tx=0x7f42cc00d6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36834478edfc7131 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f38caffd700 1 -- 172.21.15.79:0/786404955 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400e070 con 0x7f38cc0ccfe0 2023-06-08T03:50:44.735 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f38caffd700 1 -- 172.21.15.79:0/786404955 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c400bb20 con 0x7f38cc0ccfe0 2023-06-08T03:50:44.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f38caffd700 1 -- 172.21.15.79:0/786404955 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4012430 con 0x7f38cc0ccfe0 2023-06-08T03:50:44.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f42d159a700 1 -- 172.21.15.79:0/200769170 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc010070 con 0x7f42d40ca160 2023-06-08T03:50:44.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f42d159a700 1 -- 172.21.15.79:0/200769170 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc00dc80 con 0x7f42d40ca160 2023-06-08T03:50:44.736 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.722+0000 7f42d159a700 1 -- 172.21.15.79:0/200769170 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc007b30 con 0x7f42d40ca160 2023-06-08T03:50:44.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f38d283c700 1 -- 172.21.15.79:0/786404955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 msgr2=0x7f38cc0cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f38d283c700 1 --2- 172.21.15.79:0/786404955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 0x7f38cc0cd460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009d90 tx=0x7f38c400b330 comp rx=0 tx=0).stop 2023-06-08T03:50:44.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f38d283c700 1 -- 172.21.15.79:0/786404955 shutdown_connections 2023-06-08T03:50:44.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f38d283c700 1 --2- 172.21.15.79:0/786404955 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 0x7f38cc0cd460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.737 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f38d283c700 1 --2- 172.21.15.79:0/786404955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f38d283c700 1 -- 172.21.15.79:0/786404955 >> 172.21.15.79:0/786404955 conn(0x7f38cc0ce4e0 msgr2=0x7f38cc0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f42d8a14700 1 -- 172.21.15.79:0/200769170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d40ca160 msgr2=0x7f42d40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.738 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.723+0000 7f42d8a14700 1 --2- 172.21.15.79:0/200769170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d40ca160 0x7f42d40cde80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42d40cf4f0 tx=0x7f42cc00d6a0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc950007230 con 0x7fc9600b0990 2023-06-08T03:50:44.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95cff9700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc94c06ad30 0x7fc94c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc950089ae0 con 0x7fc9600b0990 2023-06-08T03:50:44.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95cff9700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] conn(0x7fc94c074a00 0x7fc94c076e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 --> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc94c077500 con 0x7fc94c074a00 2023-06-08T03:50:44.739 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95ffff700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc94c06ad30 0x7fc94c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.724+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc950089f80 con 0x7fc9600b0990 2023-06-08T03:50:44.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.725+0000 7f38d283c700 1 -- 172.21.15.79:0/786404955 shutdown_connections 2023-06-08T03:50:44.740 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.725+0000 7f38d283c700 1 -- 172.21.15.79:0/786404955 wait complete. 2023-06-08T03:50:44.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.725+0000 7fc95ffff700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc94c06ad30 0x7fc94c06d1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc9600d07f0 tx=0x7fc958010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.726+0000 7fc95f7fe700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] conn(0x7fc94c074a00 0x7fc94c076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.726+0000 7f38d283c700 1 Processor -- start 2023-06-08T03:50:44.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.726+0000 7fc95f7fe700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] conn(0x7fc94c074a00 0x7fc94c076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.726+0000 7f38d283c700 1 -- start start 2023-06-08T03:50:44.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.726+0000 7fbde4352700 1 Processor -- start 2023-06-08T03:50:44.742 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7fbde4352700 1 -- start start 2023-06-08T03:50:44.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== osd.4 v2:172.21.15.156:6800/1078658928 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc94c077500 con 0x7fc94c074a00 2023-06-08T03:50:44.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f42d8a14700 1 -- 172.21.15.79:0/200769170 shutdown_connections 2023-06-08T03:50:44.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f42d8a14700 1 --2- 172.21.15.79:0/200769170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d40ca160 0x7f42d40cde80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f42d8a14700 1 --2- 172.21.15.79:0/200769170 >> v2:172.21.15.79:3300/0 conn(0x7f42d40af8d0 0x7f42d40713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f42d8a14700 1 -- 172.21.15.79:0/200769170 >> 172.21.15.79:0/200769170 conn(0x7f42d40ce4e0 msgr2=0x7f42d40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f38d283c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f38d283c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38cc0cb890 0x7f38cc06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f38d283c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 0x7f38cc06c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f38d283c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc151f30 con 0x7f38cc0ccfe0 2023-06-08T03:50:44.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f38d283c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc097f20 con 0x7f38cc0cb890 2023-06-08T03:50:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.727+0000 7f38d283c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc097c40 con 0x7f38cc0c14b0 2023-06-08T03:50:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc1500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49296/0 (socket says 172.21.15.79:49296) 2023-06-08T03:50:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 -- 172.21.15.79:0/4088486603 learned_addr learned my addr 172.21.15.79:0/4088486603 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde4352700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbddc0d0740 0x7fbddc0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde4352700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbddc0d0b10 0x7fbddc0d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde4352700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde4352700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc0b4f40 con 0x7fbddc0d0b10 2023-06-08T03:50:44.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde4352700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc0d2e20 con 0x7fbddc0d0740 2023-06-08T03:50:44.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde4352700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0d3100 con 0x7fbddc0ced20 2023-06-08T03:50:44.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 -- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38cc0cb890 msgr2=0x7f38cc06c0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-08T03:50:44.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38cc0cb890 0x7f38cc06c0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 -- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 msgr2=0x7f38cc06c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0ccfe0 0x7f38cc06c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 -- 172.21.15.79:0/4088486603 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4009a70 con 0x7f38cc0c14b0 2023-06-08T03:50:44.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde20ee700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbddc0d0740 0x7fbddc0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7f38cbfff700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc1500e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f38c000d7d0 tx=0x7f38c000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde28ef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde28ef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0cf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49304/0 (socket says 172.21.15.79:49304) 2023-06-08T03:50:44.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde20ee700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbddc0d0740 0x7fbddc0cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:57872/0 (socket says 172.21.15.79:57872) 2023-06-08T03:50:44.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.728+0000 7fbde28ef700 1 -- 172.21.15.79:0/946526966 learned_addr learned my addr 172.21.15.79:0/946526966 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00125f0 con 0x7f38cc0c14b0 2023-06-08T03:50:44.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde28ef700 1 -- 172.21.15.79:0/946526966 >> v2:172.21.15.156:3300/0 conn(0x7fbddc0d0740 msgr2=0x7fbddc0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde28ef700 1 --2- 172.21.15.79:0/946526966 >> v2:172.21.15.156:3300/0 conn(0x7fbddc0d0740 0x7fbddc0cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde28ef700 1 -- 172.21.15.79:0/946526966 >> v2:172.21.15.79:3300/0 conn(0x7fbddc0d0b10 msgr2=0x7fbddc0d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde28ef700 1 --2- 172.21.15.79:0/946526966 >> v2:172.21.15.79:3300/0 conn(0x7fbddc0d0b10 0x7fbddc0d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde28ef700 1 -- 172.21.15.79:0/946526966 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0d33e0 con 0x7fbddc0ced20 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde28ef700 1 --2- 172.21.15.79:0/946526966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0cf1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbddc071f80 tx=0x7fbdd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=737dd2a1452945e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde10ec700 1 -- 172.21.15.79:0/946526966 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400e070 con 0x7fbddc0ced20 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde10ec700 1 -- 172.21.15.79:0/946526966 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd400b9c0 con 0x7fbddc0ced20 2023-06-08T03:50:44.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7fbde10ec700 1 -- 172.21.15.79:0/946526966 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400bd70 con 0x7fbddc0ced20 2023-06-08T03:50:44.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0012750 con 0x7f38cc0c14b0 2023-06-08T03:50:44.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0012b00 con 0x7f38cc0c14b0 2023-06-08T03:50:44.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc095270 con 0x7f38cc0c14b0 2023-06-08T03:50:44.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.729+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc095830 con 0x7f38cc0c14b0 2023-06-08T03:50:44.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.730+0000 7fbde4352700 1 -- 172.21.15.79:0/946526966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 msgr2=0x7fbddc0cf1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.730+0000 7fbde4352700 1 --2- 172.21.15.79:0/946526966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0cf1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbddc071f80 tx=0x7fbdd400b330 comp rx=0 tx=0).stop 2023-06-08T03:50:44.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.730+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f38ac000fc0 con 0x7f38cc0c14b0 2023-06-08T03:50:44.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.731+0000 7f42d8a14700 1 -- 172.21.15.79:0/200769170 shutdown_connections 2023-06-08T03:50:44.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.731+0000 7f42d8a14700 1 -- 172.21.15.79:0/200769170 wait complete. 2023-06-08T03:50:44.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fdc2fbde700 1 Processor -- start 2023-06-08T03:50:44.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38c0012c60 con 0x7f38cc0c14b0 2023-06-08T03:50:44.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fac99e65700 1 Processor -- start 2023-06-08T03:50:44.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7f38c8ff9700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38b806adb0 0x7f38b806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f38c0089820 con 0x7f38cc0c14b0 2023-06-08T03:50:44.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f38c8ff9700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] conn(0x7f38b8074a80 0x7f38b8076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 --> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38b8077580 con 0x7f38b8074a80 2023-06-08T03:50:44.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c008be40 con 0x7f38cc0c14b0 2023-06-08T03:50:44.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f42d8a14700 1 Processor -- start 2023-06-08T03:50:44.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fbde4352700 1 -- 172.21.15.79:0/946526966 shutdown_connections 2023-06-08T03:50:44.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fbde4352700 1 --2- 172.21.15.79:0/946526966 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0cf1a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fbde4352700 1 --2- 172.21.15.79:0/946526966 >> v2:172.21.15.156:3300/0 conn(0x7fbddc0d0740 0x7fbddc0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fbde4352700 1 --2- 172.21.15.79:0/946526966 >> v2:172.21.15.79:3300/0 conn(0x7fbddc0d0b10 0x7fbddc0d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fbde4352700 1 -- 172.21.15.79:0/946526966 >> 172.21.15.79:0/946526966 conn(0x7fbddc071480 msgr2=0x7fbddc071850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f38cb7fe700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38b806adb0 0x7f38b806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fdc2fbde700 1 -- start start 2023-06-08T03:50:44.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f38d0dd9700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] conn(0x7f38b8074a80 0x7f38b8076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.732+0000 7fac99e65700 1 -- start start 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac99e65700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b85f0 0x7fac940d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac99e65700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fac940b59c0 0x7fac940cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac99e65700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac940b2420 con 0x7fac940d1e50 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac99e65700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940d1660 con 0x7fac940b59c0 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac99e65700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac940d1850 con 0x7fac940b85f0 2023-06-08T03:50:44.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac937fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fac940d1e50 0x7fac940d2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60626/0 (socket says 172.21.15.79:60626) 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac937fe700 1 -- 172.21.15.79:0/1303385913 learned_addr learned my addr 172.21.15.79:0/1303385913 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7fac93fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fac940b59c0 0x7fac940cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac92ffd700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b85f0 0x7fac940d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac927fc700 1 -- 172.21.15.79:0/1303385913 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140506617 0 0) 0x7fac940b2420 con 0x7fac940d1e50 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac927fc700 1 -- 172.21.15.79:0/1303385913 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac7c0035b0 con 0x7fac940d1e50 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac93fff700 1 -- 172.21.15.79:0/1303385913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b85f0 msgr2=0x7fac940d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac93fff700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b85f0 0x7fac940d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac93fff700 1 -- 172.21.15.79:0/1303385913 >> v1:172.21.15.156:6789/0 conn(0x7fac940d1e50 legacy=0x7fac940d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac93fff700 1 -- 172.21.15.79:0/1303385913 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940ce760 con 0x7fac940b59c0 2023-06-08T03:50:44.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac92ffd700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b85f0 0x7fac940d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:44.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.733+0000 7f42d8a14700 1 -- start start 2023-06-08T03:50:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d8a14700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d4071920 0x7f42d4098530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d8a14700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42d40af8d0 0x7f42d40b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d8a14700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42d40ca160 0x7f42d40b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d8a14700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d415b7b0 con 0x7f42d40af8d0 2023-06-08T03:50:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d8a14700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c74a0 con 0x7f42d4071920 2023-06-08T03:50:44.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d8a14700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c7780 con 0x7f42d40ca160 2023-06-08T03:50:44.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f38d0dd9700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] conn(0x7f38b8074a80 0x7f38b8076eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fdc2fbde700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc280ca160 0x7fdc280cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fdc2fbde700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdc28071af0 0x7fdc280c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fdc2fbde700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280cda10 con 0x7fdc280af8d0 2023-06-08T03:50:44.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fdc2fbde700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc280ca530 con 0x7fdc28071af0 2023-06-08T03:50:44.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fdc2fbde700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280cb640 con 0x7fdc280ca160 2023-06-08T03:50:44.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7fac93fff700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940b59c0 0x7fac940cf520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac84009840 tx=0x7fac84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11b3f07632475a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fac927fc700 1 -- 172.21.15.79:0/1303385913 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84011830 con 0x7fac940b59c0 2023-06-08T03:50:44.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fdc2d179700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdc280af8d0 0x7fdc280713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58476/0 (socket says 172.21.15.79:58476) 2023-06-08T03:50:44.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fdc2d179700 1 -- 172.21.15.79:0/209390055 learned_addr learned my addr 172.21.15.79:0/209390055 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:44.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fac927fc700 1 -- 172.21.15.79:0/1303385913 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac84011990 con 0x7fac940b59c0 2023-06-08T03:50:44.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fac927fc700 1 -- 172.21.15.79:0/1303385913 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84011d20 con 0x7fac940b59c0 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42d40af8d0 0x7f42d40b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42d40af8d0 0x7f42d40b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34996/0 (socket says 172.21.15.79:34996) 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.734+0000 7f42d1d9b700 1 -- 172.21.15.79:0/1254627593 learned_addr learned my addr 172.21.15.79:0/1254627593 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42d1d9b700 1 -- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42d40ca160 msgr2=0x7f42d40b3f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42d1d9b700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42d40ca160 0x7f42d40b3f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42d1d9b700 1 -- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d4071920 msgr2=0x7f42d4098530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:44.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42d1d9b700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42d4071920 0x7f42d4098530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42d1d9b700 1 -- 172.21.15.79:0/1254627593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc00d050 con 0x7f42d40af8d0 2023-06-08T03:50:44.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42d1d9b700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42d40af8d0 0x7f42d40b3a30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f42c400e870 tx=0x7f42c400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fbde4352700 1 -- 172.21.15.79:0/946526966 shutdown_connections 2023-06-08T03:50:44.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/209390055 >> v2:172.21.15.156:3300/0 conn(0x7fdc28071af0 0x7fdc280c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718957112 0 0) 0x7fdc280cda10 con 0x7fdc280af8d0 2023-06-08T03:50:44.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc140035b0 con 0x7fdc280af8d0 2023-06-08T03:50:44.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7fdc2d97a700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc280ca160 0x7fdc280cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400ee70 con 0x7f42d40af8d0 2023-06-08T03:50:44.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c400c850 con 0x7f42d40af8d0 2023-06-08T03:50:44.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400cbc0 con 0x7f42d40af8d0 2023-06-08T03:50:44.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d40c7a60 con 0x7f42d40af8d0 2023-06-08T03:50:44.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d4098a70 con 0x7f42d40af8d0 2023-06-08T03:50:44.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42b8000fc0 con 0x7f42d40af8d0 2023-06-08T03:50:44.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.738+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42c400cd20 con 0x7f42d40af8d0 2023-06-08T03:50:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 -- 172.21.15.79:0/1303385913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940b59c0 msgr2=0x7fac940cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940b59c0 0x7fac940cf520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac84009840 tx=0x7fac84009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 -- 172.21.15.79:0/1303385913 shutdown_connections 2023-06-08T03:50:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940b59c0 0x7fac940cf520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 --2- 172.21.15.79:0/1303385913 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b85f0 0x7fac940d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 -- 172.21.15.79:0/1303385913 >> 172.21.15.79:0/1303385913 conn(0x7fac94071480 msgr2=0x7fac94071850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 -- 172.21.15.79:0/1303385913 shutdown_connections 2023-06-08T03:50:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fac99e65700 1 -- 172.21.15.79:0/1303385913 wait complete. 2023-06-08T03:50:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7fac99e65700 1 Processor -- start 2023-06-08T03:50:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.735+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== osd.2 v2:172.21.15.79:6808/4041064949 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f38b8077580 con 0x7f38b8074a80 2023-06-08T03:50:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672305383 0 0) 0x7fdc140035b0 con 0x7fdc280af8d0 2023-06-08T03:50:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc280ca160 msgr2=0x7fdc280cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc280ca160 0x7fdc280cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 >> v2:172.21.15.156:3300/0 conn(0x7fdc28071af0 msgr2=0x7fdc280c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 --2- 172.21.15.79:0/209390055 >> v2:172.21.15.156:3300/0 conn(0x7fdc28071af0 0x7fdc280c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280af540 con 0x7fdc280af8d0 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fdc18004d40 con 0x7fdc280af8d0 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 >> v1:172.21.15.79:6789/0 conn(0x7fdc280af8d0 legacy=0x7fdc280713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc14003a00 0x7fdc14003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14004850 0x7fdc14004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc1400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280ca530 con 0x7fdc14004850 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280cb640 con 0x7fdc14003a00 2023-06-08T03:50:44.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1400c0b0 con 0x7fdc14005200 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc1400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.736+0000 7fdc2d179700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14004850 0x7fdc14004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7fdc2e17b700 1 -- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc14003a00 msgr2=0x7fdc14003e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc14003a00 0x7fdc14003e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7fdc2e17b700 1 -- 172.21.15.79:0/209390055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14004850 msgr2=0x7fdc14004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc14004850 0x7fdc14004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.737+0000 7fdc2e17b700 1 -- 172.21.15.79:0/209390055 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280af540 con 0x7fdc14005200 2023-06-08T03:50:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7f42c2ffd700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f42b406ad60 0x7f42b406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.738+0000 7fac99e65700 1 -- start start 2023-06-08T03:50:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc1400ba10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009f90 tx=0x7fdc14005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb725f4095d14dca server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f42c4089940 con 0x7f42d40af8d0 2023-06-08T03:50:44.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7f42c2ffd700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] conn(0x7f42b4074a30 0x7f42b4076e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 --> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42b4077530 con 0x7f42b4074a30 2023-06-08T03:50:44.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42c4089de0 con 0x7f42d40af8d0 2023-06-08T03:50:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fbde4352700 1 -- 172.21.15.79:0/946526966 wait complete. 2023-06-08T03:50:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac99e65700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b59c0 0x7fac94146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac99e65700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac940b85f0 0x7fac941634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac99e65700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940d1e50 0x7fac94161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac99e65700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac94148760 con 0x7fac940d1e50 2023-06-08T03:50:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac99e65700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac94146990 con 0x7fac940b85f0 2023-06-08T03:50:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac99e65700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac94160e40 con 0x7fac940b59c0 2023-06-08T03:50:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7f42d259c700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f42b406ad60 0x7f42b406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac93fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940d1e50 0x7fac94161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac93fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940d1e50 0x7fac94161520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35012/0 (socket says 172.21.15.79:35012) 2023-06-08T03:50:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.739+0000 7fac93fff700 1 -- 172.21.15.79:0/436370004 learned_addr learned my addr 172.21.15.79:0/436370004 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac92ffd700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac940b85f0 0x7fac941634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac937fe700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b59c0 0x7fac94146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7f42d2d9d700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] conn(0x7f42b4074a30 0x7f42b4076e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7f42d259c700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f42b406ad60 0x7f42b406d210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00d6e0 tx=0x7f42cc012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7f42d2d9d700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] conn(0x7f42b4074a30 0x7f42b4076e60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac93fff700 1 -- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b59c0 msgr2=0x7fac94146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac93fff700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac940b59c0 0x7fac94146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac93fff700 1 -- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac940b85f0 msgr2=0x7fac941634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac93fff700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac940b85f0 0x7fac941634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.740+0000 7fac93fff700 1 -- 172.21.15.79:0/436370004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840094a0 con 0x7fac940d1e50 2023-06-08T03:50:44.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 Processor -- start 2023-06-08T03:50:44.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fac93fff700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940d1e50 0x7fac94161520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac84009b90 tx=0x7fac84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 -- start start 2023-06-08T03:50:44.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbddc0d0b10 0x7fbddc0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbddc1f5cc0 0x7fbddc1f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0cb5f0 con 0x7fbddc0d0b10 2023-06-08T03:50:44.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0c77a0 con 0x7fbddc1f5cc0 2023-06-08T03:50:44.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.741+0000 7fbde4352700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0c7a50 con 0x7fbddc0ced20 2023-06-08T03:50:44.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac840036a0 con 0x7fac940d1e50 2023-06-08T03:50:44.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8401f070 con 0x7fac940d1e50 2023-06-08T03:50:44.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84003a40 con 0x7fac940d1e50 2023-06-08T03:50:44.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== osd.0 v2:172.21.15.79:6800/233623149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f42b4077530 con 0x7f42b4074a30 2023-06-08T03:50:44.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac94161120 con 0x7fac940d1e50 2023-06-08T03:50:44.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac941483f0 con 0x7fac940d1e50 2023-06-08T03:50:44.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac940ced20 con 0x7fac940d1e50 2023-06-08T03:50:44.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.744+0000 7f38cb7fe700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38b806adb0 0x7f38b806d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007fd0 tx=0x7f38bc007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.744+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2000ec30 con 0x7fdc14005200 2023-06-08T03:50:44.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.744+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2000f040 con 0x7fdc14005200 2023-06-08T03:50:44.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.744+0000 7fdc2c978700 1 -- 172.21.15.79:0/209390055 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20014480 con 0x7fdc14005200 2023-06-08T03:50:44.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.744+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac84003ba0 con 0x7fac940d1e50 2023-06-08T03:50:44.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2fbde700 1 -- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 msgr2=0x7fdc1400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2fbde700 1 --2- 172.21.15.79:0/209390055 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc1400ba10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009f90 tx=0x7fdc14005880 comp rx=0 tx=0).stop 2023-06-08T03:50:44.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2e17b700 1 -- 172.21.15.79:0/209390055 reap_dead start 2023-06-08T03:50:44.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2fbde700 1 -- 172.21.15.79:0/209390055 shutdown_connections 2023-06-08T03:50:44.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2fbde700 1 -- 172.21.15.79:0/209390055 >> 172.21.15.79:0/209390055 conn(0x7fdc280ce4e0 msgr2=0x7fdc280ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fac7bfff700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fac8006ace0 0x7fac8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2fbde700 1 -- 172.21.15.79:0/209390055 shutdown_connections 2023-06-08T03:50:44.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.745+0000 7fdc2fbde700 1 -- 172.21.15.79:0/209390055 wait complete. 2023-06-08T03:50:44.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.747+0000 7fac937fe700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fac8006ace0 0x7fac8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.747+0000 7fdc2fbde700 1 Processor -- start 2023-06-08T03:50:44.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.747+0000 7fdc2fbde700 1 -- start start 2023-06-08T03:50:44.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fac937fe700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fac8006ace0 0x7fac8006d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001560 tx=0x7fac8c0019f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fdc2fbde700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc280b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fdc2fbde700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc281ee540 0x7fdc281596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fdc2fbde700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2809a4a0 0x7fdc2809a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fdc2fbde700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2809ae60 con 0x7fdc281ee540 2023-06-08T03:50:44.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fdc2fbde700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280b3e60 con 0x7fdc2809a4a0 2023-06-08T03:50:44.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.748+0000 7fdc2fbde700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280b1760 con 0x7fdc14005200 2023-06-08T03:50:44.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fbde20ee700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fbde20ee700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49342/0 (socket says 172.21.15.79:49342) 2023-06-08T03:50:44.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fbde20ee700 1 -- 172.21.15.79:0/1287430665 learned_addr learned my addr 172.21.15.79:0/1287430665 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.742+0000 7fbde28ef700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbddc1f5cc0 0x7fbddc1f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fbde20ee700 1 -- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbddc1f5cc0 msgr2=0x7fbddc1f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fbde20ee700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbddc1f5cc0 0x7fbddc1f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fbde20ee700 1 -- 172.21.15.79:0/1287430665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbddc0d0b10 msgr2=0x7fbddc0c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fbde20ee700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbddc0d0b10 0x7fbddc0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fbde20ee700 1 -- 172.21.15.79:0/1287430665 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd4009b30 con 0x7fbddc0ced20 2023-06-08T03:50:44.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.743+0000 7fbde20ee700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0c6d20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00e810 tx=0x7fbdcc00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc280b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc280b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49350/0 (socket says 172.21.15.79:49350) 2023-06-08T03:50:44.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 -- 172.21.15.79:0/1471436145 learned_addr learned my addr 172.21.15.79:0/1471436145 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2809a4a0 0x7fdc2809a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 -- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2809a4a0 msgr2=0x7fdc2809a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdc2809a4a0 0x7fdc2809a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 -- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc281ee540 msgr2=0x7fdc281596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc281ee540 0x7fdc281596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 -- 172.21.15.79:0/1471436145 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280b1a40 con 0x7fdc14005200 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2d97a700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc280b2800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400d6a0 tx=0x7fdc2400d9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400b7f0 con 0x7fdc14005200 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2400b950 con 0x7fdc14005200 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc280b1d20 con 0x7fdc14005200 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400bd00 con 0x7fdc14005200 2023-06-08T03:50:44.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.749+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc280b22e0 con 0x7fdc14005200 2023-06-08T03:50:44.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.750+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc280cbb70 con 0x7fdc14005200 2023-06-08T03:50:44.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.751+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00c070 con 0x7fbddc0ced20 2023-06-08T03:50:44.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.751+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc013630 con 0x7fbddc0ced20 2023-06-08T03:50:44.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.751+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0139e0 con 0x7fbddc0ced20 2023-06-08T03:50:44.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fac84021070 con 0x7fac940d1e50 2023-06-08T03:50:44.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fac7bfff700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] conn(0x7fac800748b0 0x7fac80076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 --> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac800773b0 con 0x7fac800748b0 2023-06-08T03:50:44.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac84055240 con 0x7fac940d1e50 2023-06-08T03:50:44.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc2401f440 con 0x7fdc14005200 2023-06-08T03:50:44.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fdc1e7fc700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdc0c06ace0 0x7fdc0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdc24089720 con 0x7fdc14005200 2023-06-08T03:50:44.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fdc1e7fc700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] conn(0x7fdc0c074930 0x7fdc0c076d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 --> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc0c077430 con 0x7fdc0c074930 2023-06-08T03:50:44.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc24089bc0 con 0x7fdc14005200 2023-06-08T03:50:44.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.753+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] conn(0x7fdc0c074930 0x7fdc0c076d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc0c7d30 con 0x7fbddc0ced20 2023-06-08T03:50:44.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.752+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc0cb180 con 0x7fbddc0ced20 2023-06-08T03:50:44.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.753+0000 7fdc2d179700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdc0c06ace0 0x7fdc0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.753+0000 7fdc2e17b700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] conn(0x7fdc0c074930 0x7fdc0c076d60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.753+0000 7fdc2d179700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdc0c06ace0 0x7fdc0c06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc180048d0 tx=0x7fdc18001080 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.754+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdcc013b40 con 0x7fbddc0ced20 2023-06-08T03:50:44.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.754+0000 7fbdd2ffd700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbdc006ad30 0x7fbdc006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.754+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbdcc089110 con 0x7fbddc0ced20 2023-06-08T03:50:44.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.754+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== osd.3 v2:172.21.15.79:6824/481813105 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdc0c077430 con 0x7fdc0c074930 2023-06-08T03:50:44.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.754+0000 7fbde18ed700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbdc006ad30 0x7fbdc006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.755+0000 7fbde18ed700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbdc006ad30 0x7fbdc006d1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbddc071450 tx=0x7fbdd8006d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.756+0000 7fac92ffd700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] conn(0x7fac800748b0 0x7fac80076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.756+0000 7f4cb8579700 1 Processor -- start 2023-06-08T03:50:44.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.756+0000 7fac92ffd700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] conn(0x7fac800748b0 0x7fac80076ce0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.756+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 --> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc9600cd5d0 con 0x7fc94c074a00 2023-06-08T03:50:44.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.756+0000 7f4cb8579700 1 -- start start 2023-06-08T03:50:44.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.757+0000 7fc95cff9700 1 -- 172.21.15.79:0/1703124236 <== osd.4 v2:172.21.15.156:6800/1078658928 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc9540015c0 con 0x7fc94c074a00 2023-06-08T03:50:44.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.757+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] conn(0x7fc94c074a00 msgr2=0x7fc94c076e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.757+0000 7fc96697c700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6800/1078658928,v1:172.21.15.156:6801/1078658928] conn(0x7fc94c074a00 0x7fc94c076e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc94c06ad30 msgr2=0x7fc94c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc94c06ad30 0x7fc94c06d1e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc9600d07f0 tx=0x7fc958010000 comp rx=0 tx=0).stop 2023-06-08T03:50:44.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9600b0990 msgr2=0x7fc9600b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 --2- 172.21.15.79:0/1703124236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9600b0990 0x7fc9600b0e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc960072530 tx=0x7fc95000c710 comp rx=0 tx=0).stop 2023-06-08T03:50:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc964f19700 1 -- 172.21.15.79:0/1703124236 reap_dead start 2023-06-08T03:50:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 shutdown_connections 2023-06-08T03:50:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 >> 172.21.15.79:0/1703124236 conn(0x7fc960069d50 msgr2=0x7fc9600b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== osd.1 v2:172.21.15.79:6816/4207501459 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fac800773b0 con 0x7fac800748b0 2023-06-08T03:50:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb8579700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 0x7f4cb00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb8579700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4cb00cde80 0x7f4cb00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb8579700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb00cda10 con 0x7f4cb00713e0 2023-06-08T03:50:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb8579700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb00cb640 con 0x7f4cb00cde80 2023-06-08T03:50:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb8579700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00afe10 con 0x7f4cb00ceff0 2023-06-08T03:50:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 shutdown_connections 2023-06-08T03:50:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fc96697c700 1 -- 172.21.15.79:0/1703124236 wait complete. 2023-06-08T03:50:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 0x7f4cb00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 0x7f4cb00ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49366/0 (socket says 172.21.15.79:49366) 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 -- 172.21.15.79:0/1876580985 learned_addr learned my addr 172.21.15.79:0/1876580985 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb5b14700 1 --2- 172.21.15.79:0/1876580985 >> v2:172.21.15.79:3300/0 conn(0x7f4cb00cde80 0x7f4cb00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 -- 172.21.15.79:0/1876580985 >> v2:172.21.15.79:3300/0 conn(0x7f4cb00cde80 msgr2=0x7f4cb00af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 --2- 172.21.15.79:0/1876580985 >> v2:172.21.15.79:3300/0 conn(0x7f4cb00cde80 0x7f4cb00af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 -- 172.21.15.79:0/1876580985 >> v1:172.21.15.79:6789/0 conn(0x7f4cb00713e0 legacy=0x7f4cb00c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7f4cb6315700 1 -- 172.21.15.79:0/1876580985 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00ce2f0 con 0x7f4cb00ceff0 2023-06-08T03:50:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fbde4352700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] conn(0x7fbddc0b6450 0x7fbddc0607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.758+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 --> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbddc0d36c0 con 0x7fbddc0b6450 2023-06-08T03:50:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.759+0000 7f4cb6315700 1 --2- 172.21.15.79:0/1876580985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 0x7f4cb00ca160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009ea0 tx=0x7f4ca800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e857e38c22f1f71 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.759+0000 7fbde28ef700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] conn(0x7fbddc0b6450 0x7fbddc0607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.759+0000 7fbde28ef700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] conn(0x7fbddc0b6450 0x7fbddc0607d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.760+0000 7f4cb5313700 1 -- 172.21.15.79:0/1876580985 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800e070 con 0x7f4cb00ceff0 2023-06-08T03:50:44.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.760+0000 7f4cb5313700 1 -- 172.21.15.79:0/1876580985 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca800b980 con 0x7f4cb00ceff0 2023-06-08T03:50:44.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.760+0000 7f4cb5313700 1 -- 172.21.15.79:0/1876580985 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800bcf0 con 0x7f4cb00ceff0 2023-06-08T03:50:44.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.760+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== osd.7 v2:172.21.15.156:6816/3505887928 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbdd40099b0 con 0x7fbddc0b6450 2023-06-08T03:50:44.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.760+0000 7f4cb8579700 1 -- 172.21.15.79:0/1876580985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 msgr2=0x7f4cb00ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.760+0000 7f4cb8579700 1 --2- 172.21.15.79:0/1876580985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 0x7f4cb00ca160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009ea0 tx=0x7f4ca800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 --> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f38ac002d40 con 0x7f38b8074a80 2023-06-08T03:50:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f38c8ff9700 1 -- 172.21.15.79:0/4088486603 <== osd.2 v2:172.21.15.79:6808/4041064949 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f38ac002d40 con 0x7f38b8074a80 2023-06-08T03:50:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f4cb8579700 1 -- 172.21.15.79:0/1876580985 shutdown_connections 2023-06-08T03:50:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f4cb8579700 1 --2- 172.21.15.79:0/1876580985 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00ceff0 0x7f4cb00ca160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f4cb8579700 1 --2- 172.21.15.79:0/1876580985 >> v2:172.21.15.79:3300/0 conn(0x7f4cb00cde80 0x7f4cb00af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f4cb8579700 1 -- 172.21.15.79:0/1876580985 >> 172.21.15.79:0/1876580985 conn(0x7f4cb00ce4e0 msgr2=0x7f4cb00ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f4cb8579700 1 -- 172.21.15.79:0/1876580985 shutdown_connections 2023-06-08T03:50:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.767+0000 7f4cb8579700 1 -- 172.21.15.79:0/1876580985 wait complete. 2023-06-08T03:50:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.769+0000 7f4cb8579700 1 Processor -- start 2023-06-08T03:50:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.769+0000 7f4cb8579700 1 -- start start 2023-06-08T03:50:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 --> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42b8002d40 con 0x7f42b4074a30 2023-06-08T03:50:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb8579700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00713e0 0x7f4cb0158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb8579700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb00cde80 0x7f4cb00b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb8579700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cb00ceff0 0x7f4cb00b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb8579700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00b4290 con 0x7f4cb00ceff0 2023-06-08T03:50:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb8579700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00b1980 con 0x7f4cb00cde80 2023-06-08T03:50:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb8579700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00b1c60 con 0x7f4cb00713e0 2023-06-08T03:50:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] conn(0x7f38b8074a80 msgr2=0x7f38b8076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.79:6808/4041064949,v1:172.21.15.79:6809/4041064949] conn(0x7f38b8074a80 0x7f38b8076eb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb5b14700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb00cde80 0x7f4cb00b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb5b14700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb00cde80 0x7f4cb00b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:57946/0 (socket says 172.21.15.79:57946) 2023-06-08T03:50:44.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb5b14700 1 -- 172.21.15.79:0/3283893026 learned_addr learned my addr 172.21.15.79:0/3283893026 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38b806adb0 msgr2=0x7f38b806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38b806adb0 0x7f38b806d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38bc007fd0 tx=0x7f38bc007980 comp rx=0 tx=0).stop 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 msgr2=0x7f38cc1500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 --2- 172.21.15.79:0/4088486603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38cc0c14b0 0x7f38cc1500e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f38c000d7d0 tx=0x7f38c000db90 comp rx=0 tx=0).stop 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d0dd9700 1 -- 172.21.15.79:0/4088486603 reap_dead start 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 shutdown_connections 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 >> 172.21.15.79:0/4088486603 conn(0x7f38cc0ce4e0 msgr2=0x7f38cc15fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 shutdown_connections 2023-06-08T03:50:44.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f38d283c700 1 -- 172.21.15.79:0/4088486603 wait complete. 2023-06-08T03:50:44.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb5b14700 1 -- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00713e0 msgr2=0x7f4cb0158410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.770+0000 7f4cb5b14700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cb00713e0 0x7f4cb0158410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4cb5b14700 1 -- 172.21.15.79:0/3283893026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cb00ceff0 msgr2=0x7f4cb00b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4cb5b14700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cb00ceff0 0x7f4cb00b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4cb5b14700 1 -- 172.21.15.79:0/3283893026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8009b30 con 0x7f4cb00cde80 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4cb5b14700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb00cde80 0x7f4cb00b1310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00d800 tx=0x7f4cac00dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0125f0 con 0x7f4cb00cde80 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac012750 con 0x7f4cb00cde80 2023-06-08T03:50:44.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac012b00 con 0x7f4cb00cde80 2023-06-08T03:50:44.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb00b1f40 con 0x7f4cb00cde80 2023-06-08T03:50:44.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.771+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb00ac4b0 con 0x7f4cb00cde80 2023-06-08T03:50:44.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.772+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cb00cb890 con 0x7f4cb00cde80 2023-06-08T03:50:44.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.773+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4cac012c60 con 0x7f4cb00cde80 2023-06-08T03:50:44.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 --> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc280cb890 con 0x7fdc0c074930 2023-06-08T03:50:44.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7f4ca6ffd700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4c9406ad30 0x7f4c9406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4cac08a430 con 0x7f4cb00cde80 2023-06-08T03:50:44.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7f4ca6ffd700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] conn(0x7f4c94074a00 0x7f4c94076e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 --> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c94077500 con 0x7f4c94074a00 2023-06-08T03:50:44.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cac08a8d0 con 0x7f4cb00cde80 2023-06-08T03:50:44.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.774+0000 7f4cb6b16700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] conn(0x7f4c94074a00 0x7f4c94076e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.775+0000 7f4cb6b16700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] conn(0x7f4c94074a00 0x7f4c94076e30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.775+0000 7f42c2ffd700 1 -- 172.21.15.79:0/1254627593 <== osd.0 v2:172.21.15.79:6800/233623149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f42b8002d40 con 0x7f42b4074a30 2023-06-08T03:50:44.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.775+0000 7f84a519f700 1 Processor -- start 2023-06-08T03:50:44.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.775+0000 7f84a519f700 1 -- start start 2023-06-08T03:50:44.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== osd.6 v2:172.21.15.156:6824/3611210445 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4c94077500 con 0x7f4c94074a00 2023-06-08T03:50:44.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f84a519f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a00ca560 0x7f84a006fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f84a519f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84a006bc40 0x7f84a006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f84a519f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a006c5e0 con 0x7f84a015a010 2023-06-08T03:50:44.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f84a519f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a015a3e0 con 0x7f84a006bc40 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f84a519f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a00c89b0 con 0x7f84a00ca560 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.776+0000 7f84a499e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84a006bc40 0x7f84a006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f84a006bc40 0x7f84a006c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35060/0 (socket says 172.21.15.79:35060) 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 -- 172.21.15.79:0/3749223008 learned_addr learned my addr 172.21.15.79:0/3749223008 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 -- 172.21.15.79:0/3749223008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a00ca560 msgr2=0x7f84a006fd30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 --2- 172.21.15.79:0/3749223008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a00ca560 0x7f84a006fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 -- 172.21.15.79:0/3749223008 >> v1:172.21.15.156:6789/0 conn(0x7f84a015a010 legacy=0x7f84a00c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 -- 172.21.15.79:0/3749223008 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a00c8c50 con 0x7f84a006bc40 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f84a499e700 1 --2- 172.21.15.79:0/3749223008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a006bc40 0x7f84a006c0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8498009e50 tx=0x7f849800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad64517974b378ab server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f849effd700 1 -- 172.21.15.79:0/3749223008 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800e070 con 0x7f84a006bc40 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f849effd700 1 -- 172.21.15.79:0/3749223008 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849800b9c0 con 0x7f84a006bc40 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.777+0000 7f849effd700 1 -- 172.21.15.79:0/3749223008 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849800bd70 con 0x7f84a006bc40 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.778+0000 7f84a519f700 1 -- 172.21.15.79:0/3749223008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a006bc40 msgr2=0x7f84a006c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.778+0000 7f84a519f700 1 --2- 172.21.15.79:0/3749223008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a006bc40 0x7f84a006c0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8498009e50 tx=0x7f849800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:44.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.778+0000 7f84a519f700 1 -- 172.21.15.79:0/3749223008 shutdown_connections 2023-06-08T03:50:44.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.778+0000 7f84a519f700 1 --2- 172.21.15.79:0/3749223008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a006bc40 0x7f84a006c0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.778+0000 7f84a519f700 1 --2- 172.21.15.79:0/3749223008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a00ca560 0x7f84a006fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.778+0000 7f84a519f700 1 -- 172.21.15.79:0/3749223008 >> 172.21.15.79:0/3749223008 conn(0x7f84a00cb9d0 msgr2=0x7f84a00ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.779+0000 7f4cb6315700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4c9406ad30 0x7f4c9406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.779+0000 7f4cb6315700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4c9406ad30 0x7f4c9406d1e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b370 tx=0x7f4ca8009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] conn(0x7f42b4074a30 msgr2=0x7f42b4076e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d8a14700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:6800/233623149,v1:172.21.15.79:6801/233623149] conn(0x7f42b4074a30 0x7f42b4076e60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f42b406ad60 msgr2=0x7f42b406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d8a14700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f42b406ad60 0x7f42b406d210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00d6e0 tx=0x7f42cc012000 comp rx=0 tx=0).stop 2023-06-08T03:50:44.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42d40af8d0 msgr2=0x7f42d40b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d8a14700 1 --2- 172.21.15.79:0/1254627593 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f42d40af8d0 0x7f42d40b3a30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f42c400e870 tx=0x7f42c400ec30 comp rx=0 tx=0).stop 2023-06-08T03:50:44.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.781+0000 7f42d2d9d700 1 -- 172.21.15.79:0/1254627593 reap_dead start 2023-06-08T03:50:44.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.783+0000 7fdc1e7fc700 1 -- 172.21.15.79:0/1471436145 <== osd.3 v2:172.21.15.79:6824/481813105 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdc2000ea30 con 0x7fdc0c074930 2023-06-08T03:50:44.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.784+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] conn(0x7fdc0c074930 msgr2=0x7fdc0c076d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.784+0000 7fdc2fbde700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.79:6824/481813105,v1:172.21.15.79:6825/481813105] conn(0x7fdc0c074930 0x7fdc0c076d60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.786+0000 7f84a519f700 1 -- 172.21.15.79:0/3749223008 shutdown_connections 2023-06-08T03:50:44.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 shutdown_connections 2023-06-08T03:50:44.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 >> 172.21.15.79:0/1254627593 conn(0x7f42d40ce4e0 msgr2=0x7f42d406c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdc0c06ace0 msgr2=0x7fdc0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdc0c06ace0 0x7fdc0c06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc180048d0 tx=0x7fdc18001080 comp rx=0 tx=0).stop 2023-06-08T03:50:44.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 msgr2=0x7fdc280b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 --2- 172.21.15.79:0/1471436145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdc14005200 0x7fdc280b2800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400d6a0 tx=0x7fdc2400d9b0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2e17b700 1 -- 172.21.15.79:0/1471436145 reap_dead start 2023-06-08T03:50:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 shutdown_connections 2023-06-08T03:50:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 >> 172.21.15.79:0/1471436145 conn(0x7fdc280ce4e0 msgr2=0x7fdc2815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 shutdown_connections 2023-06-08T03:50:44.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.787+0000 7fdc2fbde700 1 -- 172.21.15.79:0/1471436145 wait complete. 2023-06-08T03:50:44.832 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2023-06-08T03:50:44.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-08T03:50:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.786+0000 7f84a519f700 1 -- 172.21.15.79:0/3749223008 wait complete. 2023-06-08T03:50:44.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.788+0000 7f84a519f700 1 Processor -- start 2023-06-08T03:50:44.834 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:44.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-08T03:50:44.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.789+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 --> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac940cea40 con 0x7fac800748b0 2023-06-08T03:50:44.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.789+0000 7f84a519f700 1 -- start start 2023-06-08T03:50:44.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a519f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a006bc40 0x7f84a00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a519f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a015a010 0x7f84a00b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a519f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84a014d120 0x7f84a014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a519f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01e0150 con 0x7f84a015a010 2023-06-08T03:50:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a519f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01df230 con 0x7f84a014d120 2023-06-08T03:50:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a519f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a01df4d0 con 0x7f84a006bc40 2023-06-08T03:50:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a015a010 0x7f84a00b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a015a010 0x7f84a00b3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35066/0 (socket says 172.21.15.79:35066) 2023-06-08T03:50:44.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac7bfff700 1 -- 172.21.15.79:0/436370004 <== osd.1 v2:172.21.15.79:6816/4207501459 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac88009520 con 0x7fac800748b0 2023-06-08T03:50:44.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 -- 172.21.15.79:0/1972357114 learned_addr learned my addr 172.21.15.79:0/1972357114 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:44.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849ffff700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a006bc40 0x7f84a00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f84a499e700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84a014d120 0x7f84a014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 -- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a006bc40 msgr2=0x7f84a00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84a006bc40 0x7f84a00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 -- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84a014d120 msgr2=0x7f84a014d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84a014d120 0x7f84a014d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 -- 172.21.15.79:0/1972357114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498009b30 con 0x7f84a015a010 2023-06-08T03:50:44.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] conn(0x7fac800748b0 msgr2=0x7fac80076ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac99e65700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:6816/4207501459,v1:172.21.15.79:6817/4207501459] conn(0x7fac800748b0 0x7fac80076ce0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 --> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbddc0d14c0 con 0x7fbddc0b6450 2023-06-08T03:50:44.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7f849f7fe700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a015a010 0x7f84a00b3530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f849400b500 tx=0x7f849400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fac8006ace0 msgr2=0x7fac8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac99e65700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fac8006ace0 0x7fac8006d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001560 tx=0x7fac8c0019f0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940d1e50 msgr2=0x7fac94161520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.790+0000 7fac99e65700 1 --2- 172.21.15.79:0/436370004 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac940d1e50 0x7fac94161520 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac84009b90 tx=0x7fac84004750 comp rx=0 tx=0).stop 2023-06-08T03:50:44.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 shutdown_connections 2023-06-08T03:50:44.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7fac93fff700 1 -- 172.21.15.79:0/436370004 reap_dead start 2023-06-08T03:50:44.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 shutdown_connections 2023-06-08T03:50:44.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 >> 172.21.15.79:0/436370004 conn(0x7fac94071480 msgr2=0x7fac9415fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494011830 con 0x7f84a015a010 2023-06-08T03:50:44.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84940119b0 con 0x7f84a015a010 2023-06-08T03:50:44.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a01df7b0 con 0x7f84a015a010 2023-06-08T03:50:44.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.792+0000 7f42d8a14700 1 -- 172.21.15.79:0/1254627593 wait complete. 2023-06-08T03:50:44.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 shutdown_connections 2023-06-08T03:50:44.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a01dfd70 con 0x7f84a015a010 2023-06-08T03:50:44.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.791+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494011d60 con 0x7f84a015a010 2023-06-08T03:50:44.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.792+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8480000fc0 con 0x7f84a015a010 2023-06-08T03:50:44.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.792+0000 7fbdd2ffd700 1 -- 172.21.15.79:0/1287430665 <== osd.7 v2:172.21.15.156:6816/3505887928 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbdd4003a20 con 0x7fbddc0b6450 2023-06-08T03:50:44.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.792+0000 7fac99e65700 1 -- 172.21.15.79:0/436370004 wait complete. 2023-06-08T03:50:44.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] conn(0x7fbddc0b6450 msgr2=0x7fbddc0607d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6816/3505887928,v1:172.21.15.156:6817/3505887928] conn(0x7fbddc0b6450 0x7fbddc0607d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbdc006ad30 msgr2=0x7fbdc006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbdc006ad30 0x7fbdc006d1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbddc071450 tx=0x7fbdd8006d20 comp rx=0 tx=0).stop 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 msgr2=0x7fbddc0c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 --2- 172.21.15.79:0/1287430665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbddc0ced20 0x7fbddc0c6d20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00e810 tx=0x7fbdcc00eb20 comp rx=0 tx=0).stop 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde28ef700 1 -- 172.21.15.79:0/1287430665 reap_dead start 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 shutdown_connections 2023-06-08T03:50:44.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 >> 172.21.15.79:0/1287430665 conn(0x7fbddc071480 msgr2=0x7fbddc15b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 shutdown_connections 2023-06-08T03:50:44.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7fbde4352700 1 -- 172.21.15.79:0/1287430665 wait complete. 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.793+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f849401a400 con 0x7f84a015a010 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.794+0000 7f849cff9700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f848c06ace0 0x7f848c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.794+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8494089d30 con 0x7f84a015a010 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.794+0000 7f849cff9700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] conn(0x7f848c074930 0x7f848c076d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.794+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 --> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f848c077430 con 0x7f848c074930 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.794+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f849408a180 con 0x7f84a015a010 2023-06-08T03:50:44.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.795+0000 7f849ffff700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f848c06ace0 0x7f848c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.795+0000 7f84a499e700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] conn(0x7f848c074930 0x7f848c076d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:44.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.795+0000 7f84a499e700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] conn(0x7f848c074930 0x7f848c076d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.795+0000 7f849ffff700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f848c06ace0 0x7f848c06d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8490007fd0 tx=0x7f8490009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:44.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.796+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== osd.5 v2:172.21.15.156:6808/353543302 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f848c077430 con 0x7f848c074930 2023-06-08T03:50:44.849 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:44.850 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:44.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-08T03:50:44.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-08T03:50:44.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.808+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 --> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4cb0097e60 con 0x7f4c94074a00 2023-06-08T03:50:44.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.808+0000 7f4ca6ffd700 1 -- 172.21.15.79:0/3283893026 <== osd.6 v2:172.21.15.156:6824/3611210445 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4cb0097e60 con 0x7f4c94074a00 2023-06-08T03:50:44.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] conn(0x7f4c94074a00 msgr2=0x7f4c94076e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6824/3611210445,v1:172.21.15.156:6825/3611210445] conn(0x7f4c94074a00 0x7f4c94076e30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4c9406ad30 msgr2=0x7f4c9406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4c9406ad30 0x7f4c9406d1e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b370 tx=0x7f4ca8009000 comp rx=0 tx=0).stop 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb00cde80 msgr2=0x7f4cb00b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 --2- 172.21.15.79:0/3283893026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cb00cde80 0x7f4cb00b1310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00d800 tx=0x7f4cac00dbc0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb6b16700 1 -- 172.21.15.79:0/3283893026 reap_dead start 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 shutdown_connections 2023-06-08T03:50:44.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 >> 172.21.15.79:0/3283893026 conn(0x7f4cb00ce4e0 msgr2=0x7f4cb01538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 shutdown_connections 2023-06-08T03:50:44.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.809+0000 7f4cb8579700 1 -- 172.21.15.79:0/3283893026 wait complete. 2023-06-08T03:50:44.854 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:44.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-08T03:50:44.855 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:44.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-08T03:50:44.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.817+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 --> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8480002d40 con 0x7f848c074930 2023-06-08T03:50:44.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.817+0000 7f849cff9700 1 -- 172.21.15.79:0/1972357114 <== osd.5 v2:172.21.15.156:6808/353543302 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8480002d40 con 0x7f848c074930 2023-06-08T03:50:44.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] conn(0x7f848c074930 msgr2=0x7f848c076d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6808/353543302,v1:172.21.15.156:6809/353543302] conn(0x7f848c074930 0x7f848c076d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f848c06ace0 msgr2=0x7f848c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f848c06ace0 0x7f848c06d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8490007fd0 tx=0x7f8490009040 comp rx=0 tx=0).stop 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a015a010 msgr2=0x7f84a00b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 --2- 172.21.15.79:0/1972357114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84a015a010 0x7f84a00b3530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f849400b500 tx=0x7f849400b810 comp rx=0 tx=0).stop 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a499e700 1 -- 172.21.15.79:0/1972357114 reap_dead start 2023-06-08T03:50:44.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 shutdown_connections 2023-06-08T03:50:44.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 >> 172.21.15.79:0/1972357114 conn(0x7f84a00cb9d0 msgr2=0x7f84a0157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:44.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 shutdown_connections 2023-06-08T03:50:44.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:44.818+0000 7f84a519f700 1 -- 172.21.15.79:0/1972357114 wait complete. 2023-06-08T03:50:44.859 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:44.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-08T03:50:44.861 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:44.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-08T03:50:45.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.094+0000 7fcef4781700 1 Processor -- start 2023-06-08T03:50:45.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.094+0000 7fcef4781700 1 -- start start 2023-06-08T03:50:45.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef4781700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fceec071af0 0x7fceec0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef4781700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec0cda10 con 0x7fceec0af8d0 2023-06-08T03:50:45.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef4781700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec0ca530 con 0x7fceec0ca160 2023-06-08T03:50:45.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef4781700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0cb640 con 0x7fceec071af0 2023-06-08T03:50:45.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef2d1e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fceec0af8d0 0x7fceec0c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58554/0 (socket says 172.21.15.79:58554) 2023-06-08T03:50:45.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef2d1e700 1 -- 172.21.15.79:0/3165393696 learned_addr learned my addr 172.21.15.79:0/3165393696 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:45.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.095+0000 7fcef251d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fceec0ca160 0x7fceec0713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60714/0 (socket says 172.21.15.79:60714) 2023-06-08T03:50:45.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018929836 0 0) 0x7fceec0ca530 con 0x7fceec0ca160 2023-06-08T03:50:45.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced80035b0 con 0x7fceec0ca160 2023-06-08T03:50:45.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955728758 0 0) 0x7fceec0cda10 con 0x7fceec0af8d0 2023-06-08T03:50:45.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceec0ca530 con 0x7fceec0af8d0 2023-06-08T03:50:45.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489720779 0 0) 0x7fceec0ca530 con 0x7fceec0af8d0 2023-06-08T03:50:45.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fceec071af0 msgr2=0x7fceec0cde80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-08T03:50:45.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fceec071af0 0x7fceec0cde80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 >> v1:172.21.15.156:6789/0 conn(0x7fceec0ca160 legacy=0x7fceec0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0af540 con 0x7fceec0af8d0 2023-06-08T03:50:45.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fcee8004f50 con 0x7fceec0af8d0 2023-06-08T03:50:45.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.096+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 >> v1:172.21.15.79:6789/0 conn(0x7fceec0af8d0 legacy=0x7fceec0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced8004c20 0x7fced8005010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80037f0 0x7fced8003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced80043a0 0x7fced800ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0cb640 con 0x7fced80043a0 2023-06-08T03:50:45.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced800c320 con 0x7fced8004c20 2023-06-08T03:50:45.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced800c4e0 con 0x7fced80037f0 2023-06-08T03:50:45.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced80043a0 0x7fced800ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 -- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80037f0 msgr2=0x7fced8003c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:45.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80037f0 0x7fced8003c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 -- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced8004c20 msgr2=0x7fced8005010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:45.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced8004c20 0x7fced8005010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 -- 172.21.15.79:0/3165393696 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0af540 con 0x7fced80043a0 2023-06-08T03:50:45.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef2d1e700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced80043a0 0x7fced800ba90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d350 tx=0x7fced8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adebe1097a59a8d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800de70 con 0x7fced80043a0 2023-06-08T03:50:45.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80105a0 con 0x7fced80043a0 2023-06-08T03:50:45.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.097+0000 7fcef151b700 1 -- 172.21.15.79:0/3165393696 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8010950 con 0x7fced80043a0 2023-06-08T03:50:45.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.098+0000 7f91fb86d700 1 Processor -- start 2023-06-08T03:50:45.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.098+0000 7f91fb86d700 1 -- start start 2023-06-08T03:50:45.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.098+0000 7fcef4781700 1 -- 172.21.15.79:0/3165393696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced80043a0 msgr2=0x7fced800ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.098+0000 7fcef4781700 1 --2- 172.21.15.79:0/3165393696 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fced80043a0 0x7fced800ba90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d350 tx=0x7fced8005880 comp rx=0 tx=0).stop 2023-06-08T03:50:45.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.098+0000 7fcef2d1e700 1 -- 172.21.15.79:0/3165393696 reap_dead start 2023-06-08T03:50:45.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7fcef4781700 1 -- 172.21.15.79:0/3165393696 shutdown_connections 2023-06-08T03:50:45.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7fcef4781700 1 -- 172.21.15.79:0/3165393696 >> 172.21.15.79:0/3165393696 conn(0x7fceec0ce4e0 msgr2=0x7fceec0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91fb86d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40c9f50 0x7f91f40ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91fb86d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f40b1330 con 0x7f91f40af340 2023-06-08T03:50:45.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91fb86d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f40cabc0 con 0x7f91f40b20d0 2023-06-08T03:50:45.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91fb86d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40b15f0 con 0x7f91f40c9f50 2023-06-08T03:50:45.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91f9609700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f91f40af340 0x7f91f40af710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58562/0 (socket says 172.21.15.79:58562) 2023-06-08T03:50:45.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91f9609700 1 -- 172.21.15.79:0/4098775031 learned_addr learned my addr 172.21.15.79:0/4098775031 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:45.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7f91f8e08700 1 --2- 172.21.15.79:0/4098775031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40c9f50 0x7f91f40ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7febd1546700 1 Processor -- start 2023-06-08T03:50:45.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.099+0000 7febd1546700 1 -- start start 2023-06-08T03:50:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.100+0000 7f91f8e08700 1 -- 172.21.15.79:0/4098775031 >> v1:172.21.15.156:6789/0 conn(0x7f91f40b20d0 legacy=0x7f91f40b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.100+0000 7f91f8e08700 1 -- 172.21.15.79:0/4098775031 >> v1:172.21.15.79:6789/0 conn(0x7f91f40af340 legacy=0x7f91f40af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.100+0000 7f91f8e08700 1 -- 172.21.15.79:0/4098775031 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40b18d0 con 0x7f91f40c9f50 2023-06-08T03:50:45.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.100+0000 7f91f8e08700 1 --2- 172.21.15.79:0/4098775031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40c9f50 0x7f91f40ca3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009870 tx=0x7f91f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1da4b957dd389f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.100+0000 7f397d6e4700 1 Processor -- start 2023-06-08T03:50:45.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.100+0000 7f397d6e4700 1 -- start start 2023-06-08T03:50:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febd1546700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7febcc0cd790 0x7febcc0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febd1546700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7febcc0963b0 0x7febcc071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febd1546700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febcc070c30 con 0x7febcc097a10 2023-06-08T03:50:45.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febd1546700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febcc0afd70 con 0x7febcc0963b0 2023-06-08T03:50:45.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febd1546700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febcc0aed10 con 0x7febcc0cd790 2023-06-08T03:50:45.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7febcc0cd790 0x7febcc0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7febcc0cd790 0x7febcc0cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:57984/0 (socket says 172.21.15.79:57984) 2023-06-08T03:50:45.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 -- 172.21.15.79:0/629950854 learned_addr learned my addr 172.21.15.79:0/629950854 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcb7fe700 1 --2- 172.21.15.79:0/629950854 >> v2:172.21.15.79:3300/0 conn(0x7febcc0963b0 0x7febcc071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7fcef4781700 1 -- 172.21.15.79:0/3165393696 shutdown_connections 2023-06-08T03:50:45.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7fcef4781700 1 -- 172.21.15.79:0/3165393696 wait complete. 2023-06-08T03:50:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 -- 172.21.15.79:0/629950854 >> v2:172.21.15.79:3300/0 conn(0x7febcc0963b0 msgr2=0x7febcc071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 --2- 172.21.15.79:0/629950854 >> v2:172.21.15.79:3300/0 conn(0x7febcc0963b0 0x7febcc071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 -- 172.21.15.79:0/629950854 >> v1:172.21.15.79:6789/0 conn(0x7febcc097a10 legacy=0x7febcc0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 -- 172.21.15.79:0/629950854 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0ad9e0 con 0x7febcc0cd790 2023-06-08T03:50:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcb7fe700 1 --2- 172.21.15.79:0/629950854 >> v2:172.21.15.79:3300/0 conn(0x7febcc0963b0 0x7febcc071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7febcaffd700 1 --2- 172.21.15.79:0/629950854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc0cd790 0x7febcc0cdb60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febcc072450 tx=0x7febc000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a633cf6691f27d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7f397d6e4700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f397815c1f0 0x7f397806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7f397d6e4700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3978071fe0 0x7f39780cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7f397d6e4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39780cd140 con 0x7f39780cf130 2023-06-08T03:50:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7f397d6e4700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3978071110 con 0x7f3978071fe0 2023-06-08T03:50:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7f397d6e4700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3978071be0 con 0x7f397815c1f0 2023-06-08T03:50:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3978071fe0 0x7f39780cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3978071fe0 0x7f39780cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35098/0 (socket says 172.21.15.79:35098) 2023-06-08T03:50:45.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 -- 172.21.15.79:0/327222008 learned_addr learned my addr 172.21.15.79:0/327222008 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f3976ffd700 1 --2- 172.21.15.79:0/327222008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f397815c1f0 0x7f397806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7febc9ffb700 1 -- 172.21.15.79:0/629950854 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0010070 con 0x7febcc0cd790 2023-06-08T03:50:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7febc9ffb700 1 -- 172.21.15.79:0/629950854 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc0005a20 con 0x7febcc0cd790 2023-06-08T03:50:45.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 -- 172.21.15.79:0/327222008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f397815c1f0 msgr2=0x7f397806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 --2- 172.21.15.79:0/327222008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f397815c1f0 0x7f397806d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 -- 172.21.15.79:0/327222008 >> v1:172.21.15.156:6789/0 conn(0x7f39780cf130 legacy=0x7f3978069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 -- 172.21.15.79:0/327222008 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780cd390 con 0x7f3978071fe0 2023-06-08T03:50:45.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f39767fc700 1 --2- 172.21.15.79:0/327222008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3978071fe0 0x7f39780cebf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39680098f0 tx=0x7f3968009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8442b1945c2863fd server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.101+0000 7f91ebfff700 1 -- 172.21.15.79:0/4098775031 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0004230 con 0x7f91f40c9f50 2023-06-08T03:50:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f91ebfff700 1 -- 172.21.15.79:0/4098775031 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f0004390 con 0x7f91f40c9f50 2023-06-08T03:50:45.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.102+0000 7f91ebfff700 1 -- 172.21.15.79:0/4098775031 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f00036a0 con 0x7f91f40c9f50 2023-06-08T03:50:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7febc9ffb700 1 -- 172.21.15.79:0/629950854 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0005db0 con 0x7febcc0cd790 2023-06-08T03:50:45.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 -- 172.21.15.79:0/4098775031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40c9f50 msgr2=0x7f91f40ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 --2- 172.21.15.79:0/4098775031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40c9f50 0x7f91f40ca3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009870 tx=0x7f91f0009b80 comp rx=0 tx=0).stop 2023-06-08T03:50:45.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7fcef4781700 1 Processor -- start 2023-06-08T03:50:45.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f3975ffb700 1 -- 172.21.15.79:0/327222008 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800b8a0 con 0x7f3978071fe0 2023-06-08T03:50:45.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f3975ffb700 1 -- 172.21.15.79:0/327222008 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396800ba00 con 0x7f3978071fe0 2023-06-08T03:50:45.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f3975ffb700 1 -- 172.21.15.79:0/327222008 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800bd90 con 0x7f3978071fe0 2023-06-08T03:50:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 -- 172.21.15.79:0/4098775031 shutdown_connections 2023-06-08T03:50:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 --2- 172.21.15.79:0/4098775031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40c9f50 0x7f91f40ca3a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 -- 172.21.15.79:0/4098775031 >> 172.21.15.79:0/4098775031 conn(0x7f91f40ce4e0 msgr2=0x7f91f40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 -- 172.21.15.79:0/4098775031 shutdown_connections 2023-06-08T03:50:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7f91fb86d700 1 -- 172.21.15.79:0/4098775031 wait complete. 2023-06-08T03:50:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7febd1546700 1 -- 172.21.15.79:0/629950854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc0cd790 msgr2=0x7febcc0cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7febd1546700 1 --2- 172.21.15.79:0/629950854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc0cd790 0x7febcc0cdb60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7febcc072450 tx=0x7febc000d330 comp rx=0 tx=0).stop 2023-06-08T03:50:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7febd1546700 1 -- 172.21.15.79:0/629950854 shutdown_connections 2023-06-08T03:50:45.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7febd1546700 1 --2- 172.21.15.79:0/629950854 >> v2:172.21.15.79:3300/0 conn(0x7febcc0963b0 0x7febcc071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7febd1546700 1 --2- 172.21.15.79:0/629950854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc0cd790 0x7febcc0cdb60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7febd1546700 1 -- 172.21.15.79:0/629950854 >> 172.21.15.79:0/629950854 conn(0x7febcc0ce4e0 msgr2=0x7febcc0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7febd1546700 1 -- 172.21.15.79:0/629950854 shutdown_connections 2023-06-08T03:50:45.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.103+0000 7fcef4781700 1 -- start start 2023-06-08T03:50:45.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef4781700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 0x7fceec15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef4781700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceec15a7e0 0x7fceec08a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef4781700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fceec0afeb0 0x7fceec0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef4781700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec08b230 con 0x7fceec0afeb0 2023-06-08T03:50:45.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef4781700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0b0840 con 0x7fceec15a7e0 2023-06-08T03:50:45.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef4781700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec08d370 con 0x7fced80043a0 2023-06-08T03:50:45.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef251d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 0x7fceec15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef251d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 0x7fceec15ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49426/0 (socket says 172.21.15.79:49426) 2023-06-08T03:50:45.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef251d700 1 -- 172.21.15.79:0/2169914430 learned_addr learned my addr 172.21.15.79:0/2169914430 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7fcef2d1e700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fceec0afeb0 0x7fceec0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7f397d6e4700 1 -- 172.21.15.79:0/327222008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3978071fe0 msgr2=0x7f39780cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7f397d6e4700 1 --2- 172.21.15.79:0/327222008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3978071fe0 0x7f39780cebf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f39680098f0 tx=0x7f3968009c00 comp rx=0 tx=0).stop 2023-06-08T03:50:45.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.104+0000 7febd1546700 1 -- 172.21.15.79:0/629950854 wait complete. 2023-06-08T03:50:45.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef1d1c700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceec15a7e0 0x7fceec08a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef251d700 1 -- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceec15a7e0 msgr2=0x7fceec08a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef251d700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceec15a7e0 0x7fceec08a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef251d700 1 -- 172.21.15.79:0/2169914430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fceec0afeb0 msgr2=0x7fceec0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef251d700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fceec0afeb0 0x7fceec0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef251d700 1 -- 172.21.15.79:0/2169914430 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec08d650 con 0x7fced80043a0 2023-06-08T03:50:45.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef1d1c700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceec15a7e0 0x7fceec08a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:45.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f397d6e4700 1 -- 172.21.15.79:0/327222008 shutdown_connections 2023-06-08T03:50:45.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f397d6e4700 1 --2- 172.21.15.79:0/327222008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f397815c1f0 0x7f397806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f397d6e4700 1 --2- 172.21.15.79:0/327222008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3978071fe0 0x7f39780cebf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f397d6e4700 1 -- 172.21.15.79:0/327222008 >> 172.21.15.79:0/327222008 conn(0x7f39780b1d30 msgr2=0x7f397806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef2d1e700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fceec0afeb0 0x7fceec0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:45.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f91fb86d700 1 Processor -- start 2023-06-08T03:50:45.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f397d6e4700 1 -- 172.21.15.79:0/327222008 shutdown_connections 2023-06-08T03:50:45.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f91fb86d700 1 -- start start 2023-06-08T03:50:45.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f397d6e4700 1 -- 172.21.15.79:0/327222008 wait complete. 2023-06-08T03:50:45.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f5d5739c700 1 Processor -- start 2023-06-08T03:50:45.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef251d700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 0x7fceec15ad70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c460 tx=0x7fcedc00c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fc413869700 1 Processor -- start 2023-06-08T03:50:45.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc005bd0 con 0x7fced80043a0 2023-06-08T03:50:45.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc005d50 con 0x7fced80043a0 2023-06-08T03:50:45.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc007310 con 0x7fced80043a0 2023-06-08T03:50:45.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec08d930 con 0x7fced80043a0 2023-06-08T03:50:45.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7f5dc61ef700 1 Processor -- start 2023-06-08T03:50:45.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.105+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec08af20 con 0x7fced80043a0 2023-06-08T03:50:45.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f5dc61ef700 1 -- start start 2023-06-08T03:50:45.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7febd1546700 1 Processor -- start 2023-06-08T03:50:45.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f91fb86d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40af340 0x7f91f406c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f91fb86d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 0x7f91f406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f91fb86d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91f40c9f50 0x7f91f40c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f91fb86d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f414f980 con 0x7f91f40b20d0 2023-06-08T03:50:45.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f91fb86d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f414fda0 con 0x7f91f40c9f50 2023-06-08T03:50:45.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7f91fb86d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f414ed70 con 0x7f91f40af340 2023-06-08T03:50:45.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.106+0000 7febd1546700 1 -- start start 2023-06-08T03:50:45.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f397d6e4700 1 Processor -- start 2023-06-08T03:50:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f5dc61ef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5dc015c450 0x7f5dc00ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f5dc61ef700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 0x7f5dc00cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f5dc61ef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc00afd50 con 0x7f5dc00c8b30 2023-06-08T03:50:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f5dc61ef700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc00caca0 con 0x7f5dc00ad640 2023-06-08T03:50:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f5dc61ef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc00af540 con 0x7f5dc015c450 2023-06-08T03:50:45.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7febd1546700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febcc0963b0 0x7febcc0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7febd1546700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 0x7febcc206b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7febd1546700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febcc0cd790 0x7febcc1f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7febd1546700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0c5150 con 0x7febcc0cd790 2023-06-08T03:50:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7febd1546700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc08d8d0 con 0x7febcc097a10 2023-06-08T03:50:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7febd1546700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0c4f20 con 0x7febcc0963b0 2023-06-08T03:50:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 0x7f91f406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 0x7f91f406c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35108/0 (socket says 172.21.15.79:35108) 2023-06-08T03:50:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 -- 172.21.15.79:0/2894075846 learned_addr learned my addr 172.21.15.79:0/2894075846 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 -- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40af340 msgr2=0x7f91f406c120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:45.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40af340 0x7f91f406c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 -- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91f40c9f50 msgr2=0x7f91f40c3d80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:45.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91f40c9f50 0x7f91f40c3d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f91f8e08700 1 -- 172.21.15.79:0/2894075846 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00094a0 con 0x7f91f40b20d0 2023-06-08T03:50:45.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbeffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 0x7f5dc00cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbeffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 0x7f5dc00cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35118/0 (socket says 172.21.15.79:35118) 2023-06-08T03:50:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 0x7febcc206b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbeffd700 1 -- 172.21.15.79:0/243051465 learned_addr learned my addr 172.21.15.79:0/243051465 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbf7fe700 1 --2- 172.21.15.79:0/243051465 >> v2:172.21.15.156:3300/0 conn(0x7f5dc015c450 0x7f5dc00ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febcaffd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febcc0963b0 0x7febcc0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febcaffd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febcc0963b0 0x7febcc0c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49438/0 (socket says 172.21.15.79:49438) 2023-06-08T03:50:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febcaffd700 1 -- 172.21.15.79:0/484643521 learned_addr learned my addr 172.21.15.79:0/484643521 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 0x7febcc206b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58014/0 (socket says 172.21.15.79:58014) 2023-06-08T03:50:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7fc413869700 1 -- start start 2023-06-08T03:50:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.107+0000 7f5d5739c700 1 -- start start 2023-06-08T03:50:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5d5739c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500cef80 0x7f5d5006c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5d5739c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5015cbd0 con 0x7f5d500704a0 2023-06-08T03:50:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5d5739c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5015be10 con 0x7f5d501581c0 2023-06-08T03:50:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5d5739c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50070f60 con 0x7f5d500cef80 2023-06-08T03:50:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced40052f0 con 0x7fced80043a0 2023-06-08T03:50:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/243051465 >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 msgr2=0x7f5dc00cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbf7fe700 1 --2- 172.21.15.79:0/243051465 >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 0x7f5dc00cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/243051465 >> v1:172.21.15.156:6789/0 conn(0x7f5dc00c8b30 legacy=0x7f5dc00ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/243051465 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc00af790 con 0x7f5dc015c450 2023-06-08T03:50:45.161 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbeffd700 1 --2- 172.21.15.79:0/243051465 >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 0x7f5dc00cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:45.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febcb7fe700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febcc0cd790 0x7febcc1f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.162 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc413869700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc40c0ceff0 0x7fc40c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc413869700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc40c0cde80 0x7fc40c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc413869700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0713e0 0x7fc40c071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc413869700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c0cda10 con 0x7fc40c0cde80 2023-06-08T03:50:45.163 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc413869700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40c071d80 con 0x7fc40c0ceff0 2023-06-08T03:50:45.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc413869700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c0cb640 con 0x7fc40c0713e0 2023-06-08T03:50:45.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc411e06700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0713e0 0x7fc40c071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc411e06700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0713e0 0x7fc40c071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49448/0 (socket says 172.21.15.79:49448) 2023-06-08T03:50:45.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc411e06700 1 -- 172.21.15.79:0/3425560695 learned_addr learned my addr 172.21.15.79:0/3425560695 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7fc411605700 1 --2- 172.21.15.79:0/3425560695 >> v2:172.21.15.156:3300/0 conn(0x7fc40c0ceff0 0x7fc40c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7f5dbf7fe700 1 --2- 172.21.15.79:0/243051465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc015c450 0x7f5dc00ca760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5dc006fdf0 tx=0x7f5db0009b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f64d8457dd251838 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 -- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febcc0963b0 msgr2=0x7febcc0c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febcc0963b0 0x7febcc0c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 -- 172.21.15.79:0/484643521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febcc0cd790 msgr2=0x7febcc1f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febcc0cd790 0x7febcc1f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.108+0000 7febca7fc700 1 -- 172.21.15.79:0/484643521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0008580 con 0x7febcc097a10 2023-06-08T03:50:45.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f5dbe7fc700 1 -- 172.21.15.79:0/243051465 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0005e30 con 0x7f5dc015c450 2023-06-08T03:50:45.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f5dbe7fc700 1 -- 172.21.15.79:0/243051465 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0004030 con 0x7f5dc015c450 2023-06-08T03:50:45.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f5dbe7fc700 1 -- 172.21.15.79:0/243051465 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00043c0 con 0x7f5dc015c450 2023-06-08T03:50:45.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7febca7fc700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 0x7febcc206b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febcc0b4550 tx=0x7febbc00b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f91f8e08700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 0x7f91f406c660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f91f406e1d0 tx=0x7f91f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc411605700 1 -- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0713e0 msgr2=0x7fc40c071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc411605700 1 --2- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0713e0 0x7fc40c071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc411605700 1 -- 172.21.15.79:0/3425560695 >> v2:172.21.15.79:3300/0 conn(0x7fc40c0cde80 msgr2=0x7fc40c0af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:45.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc411605700 1 --2- 172.21.15.79:0/3425560695 >> v2:172.21.15.79:3300/0 conn(0x7fc40c0cde80 0x7fc40c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc411605700 1 -- 172.21.15.79:0/3425560695 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0afe10 con 0x7fc40c0ceff0 2023-06-08T03:50:45.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f397d6e4700 1 -- start start 2023-06-08T03:50:45.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc011620 con 0x7febcc097a10 2023-06-08T03:50:45.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc411605700 1 --2- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0ceff0 0x7fc40c0ca160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0cf4f0 tx=0x7fc408009b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af126b8f0860051b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc403fff700 1 -- 172.21.15.79:0/3425560695 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408005e30 con 0x7fc40c0ceff0 2023-06-08T03:50:45.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc403fff700 1 -- 172.21.15.79:0/3425560695 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc408006030 con 0x7fc40c0ceff0 2023-06-08T03:50:45.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fc403fff700 1 -- 172.21.15.79:0/3425560695 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4080063c0 con 0x7fc40c0ceff0 2023-06-08T03:50:45.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f5d55939700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5d500704a0 0x7f5d5006c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58610/0 (socket says 172.21.15.79:58610) 2023-06-08T03:50:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7f5d55939700 1 -- 172.21.15.79:0/2312894821 learned_addr learned my addr 172.21.15.79:0/2312894821 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:45.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcedc007470 con 0x7fced80043a0 2023-06-08T03:50:45.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7fcee2ffd700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fced006ace0 0x7fced006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcedc088ab0 con 0x7fced80043a0 2023-06-08T03:50:45.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f5dc61ef700 1 -- 172.21.15.79:0/243051465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc015c450 msgr2=0x7f5dc00ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/243051465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc015c450 0x7f5dc00ca760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5dc006fdf0 tx=0x7f5db0009b00 comp rx=0 tx=0).stop 2023-06-08T03:50:45.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f5dc61ef700 1 -- 172.21.15.79:0/243051465 shutdown_connections 2023-06-08T03:50:45.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/243051465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc015c450 0x7f5dc00ca760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/243051465 >> v2:172.21.15.79:3300/0 conn(0x7f5dc00ad640 0x7f5dc00cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f5dc61ef700 1 -- 172.21.15.79:0/243051465 >> 172.21.15.79:0/243051465 conn(0x7f5dc00ce4e0 msgr2=0x7f5dc00ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859213886 0 0) 0x7f5d5015cbd0 con 0x7f5d500704a0 2023-06-08T03:50:45.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d400035b0 con 0x7f5d500704a0 2023-06-08T03:50:45.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7fcef1d1c700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fced006ace0 0x7fced006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc011780 con 0x7febcc097a10 2023-06-08T03:50:45.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc011b10 con 0x7febcc097a10 2023-06-08T03:50:45.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc1f47f0 con 0x7febcc097a10 2023-06-08T03:50:45.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.109+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc1f4480 con 0x7febcc097a10 2023-06-08T03:50:45.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc1f3fe0 con 0x7febcc097a10 2023-06-08T03:50:45.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0010590 con 0x7f91f40b20d0 2023-06-08T03:50:45.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f0010710 con 0x7f91f40b20d0 2023-06-08T03:50:45.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0010ac0 con 0x7f91f40b20d0 2023-06-08T03:50:45.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7fc413869700 1 -- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0ceff0 msgr2=0x7fc40c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7fc413869700 1 --2- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0ceff0 0x7fc40c0ca160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0cf4f0 tx=0x7fc408009b00 comp rx=0 tx=0).stop 2023-06-08T03:50:45.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 -- 172.21.15.79:0/3425560695 shutdown_connections 2023-06-08T03:50:45.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 --2- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0713e0 0x7fc40c071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 --2- 172.21.15.79:0/3425560695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0ceff0 0x7fc40c0ca160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 --2- 172.21.15.79:0/3425560695 >> v2:172.21.15.79:3300/0 conn(0x7fc40c0cde80 0x7fc40c0af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 -- 172.21.15.79:0/3425560695 >> 172.21.15.79:0/3425560695 conn(0x7fc40c0ce4e0 msgr2=0x7fc40c0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f397d6e4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 0x7f3978152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f397d6e4700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39780cf130 0x7f3978153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f397d6e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397815c1f0 0x7f3978155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f397d6e4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39781539b0 con 0x7f397815c1f0 2023-06-08T03:50:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f397d6e4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39780b3bc0 con 0x7f3978071fe0 2023-06-08T03:50:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f397d6e4700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3978153bc0 con 0x7f39780cf130 2023-06-08T03:50:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f3976ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 0x7f3978152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.182 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f3976ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 0x7f3978152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58032/0 (socket says 172.21.15.79:58032) 2023-06-08T03:50:45.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f3976ffd700 1 -- 172.21.15.79:0/2112475037 learned_addr learned my addr 172.21.15.79:0/2112475037 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.110+0000 7f39767fc700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39780cf130 0x7f3978153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f414efe0 con 0x7f91f40b20d0 2023-06-08T03:50:45.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322618651 0 0) 0x7f5d5015be10 con 0x7f5d501581c0 2023-06-08T03:50:45.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d5015cbd0 con 0x7f5d501581c0 2023-06-08T03:50:45.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fcef1d1c700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fced006ace0 0x7fced006d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fceec1f9440 tx=0x7fcee4009510 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.185 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5dc61ef700 1 -- 172.21.15.79:0/243051465 shutdown_connections 2023-06-08T03:50:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5dc61ef700 1 -- 172.21.15.79:0/243051465 wait complete. 2023-06-08T03:50:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 -- 172.21.15.79:0/3425560695 shutdown_connections 2023-06-08T03:50:45.186 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7fc413869700 1 -- 172.21.15.79:0/3425560695 wait complete. 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f39777fe700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397815c1f0 0x7f3978155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f3976ffd700 1 -- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39780cf130 msgr2=0x7f3978153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f3976ffd700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39780cf130 0x7f3978153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f3976ffd700 1 -- 172.21.15.79:0/2112475037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397815c1f0 msgr2=0x7f3978155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f3976ffd700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397815c1f0 0x7f3978155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f3976ffd700 1 -- 172.21.15.79:0/2112475037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968009550 con 0x7f3978071fe0 2023-06-08T03:50:45.187 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f3976ffd700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 0x7f3978152c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c7f0 tx=0x7f396c00cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.111+0000 7f5d54937700 1 --2- 172.21.15.79:0/2312894821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500cef80 0x7f5d5006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.112+0000 7f5d54937700 1 -- 172.21.15.79:0/2312894821 >> v1:172.21.15.156:6789/0 conn(0x7f5d501581c0 legacy=0x7f5d500ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.112+0000 7f5d54937700 1 -- 172.21.15.79:0/2312894821 >> v1:172.21.15.79:6789/0 conn(0x7f5d500704a0 legacy=0x7f5d5006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.112+0000 7f5d54937700 1 -- 172.21.15.79:0/2312894821 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5015c0f0 con 0x7f5d500cef80 2023-06-08T03:50:45.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.113+0000 7fc413869700 1 Processor -- start 2023-06-08T03:50:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.112+0000 7f5d54937700 1 --2- 172.21.15.79:0/2312894821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500cef80 0x7f5d5006c410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098d0 tx=0x7f5d3c009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9933b0ccbc6c6af3 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.113+0000 7f5dc61ef700 1 Processor -- start 2023-06-08T03:50:45.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.113+0000 7f5dc61ef700 1 -- start start 2023-06-08T03:50:45.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.113+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91f0010c20 con 0x7f91f40b20d0 2023-06-08T03:50:45.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.113+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febbc011c70 con 0x7febcc097a10 2023-06-08T03:50:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.113+0000 7fc413869700 1 -- start start 2023-06-08T03:50:45.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dc61ef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 0x7f5dc00c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7febb37fe700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7febb806ad30 0x7febb806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dc61ef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc00c8b30 0x7f5dc00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dc61ef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc015c450 0x7f5dc00c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dc61ef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc01fedd0 con 0x7f5dc00ad640 2023-06-08T03:50:45.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dc61ef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc015d820 con 0x7f5dc00c8b30 2023-06-08T03:50:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dc61ef700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc00c68a0 con 0x7f5dc015c450 2023-06-08T03:50:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 0x7f5dc00c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 0x7f5dc00c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35148/0 (socket says 172.21.15.79:35148) 2023-06-08T03:50:45.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/3202963408 learned_addr learned my addr 172.21.15.79:0/3202963408 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7febbc089a20 con 0x7febcc097a10 2023-06-08T03:50:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7febcaffd700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7febb806ad30 0x7febb806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbffff700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc015c450 0x7f5dc00c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc015c450 msgr2=0x7f5dc00c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc015c450 0x7f5dc00c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc00c8b30 msgr2=0x7f5dc00b3710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-08T03:50:45.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc00c8b30 0x7f5dc00b3710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 -- 172.21.15.79:0/3202963408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009550 con 0x7f5dc00ad640 2023-06-08T03:50:45.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc413869700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0713e0 0x7fc40c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc413869700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 0x7fc40c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc413869700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0ceff0 0x7fc40c1f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc413869700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c0c67c0 con 0x7fc40c0cde80 2023-06-08T03:50:45.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc413869700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c0b2750 con 0x7fc40c0713e0 2023-06-08T03:50:45.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc413869700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c06d000 con 0x7fc40c0ceff0 2023-06-08T03:50:45.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7f5dbf7fe700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 0x7f5dc00c6360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7febcaffd700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7febb806ad30 0x7febb806d1e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc0011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc410e04700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 0x7fc40c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc411605700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0713e0 0x7fc40c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc410e04700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 0x7fc40c0b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35154/0 (socket says 172.21.15.79:35154) 2023-06-08T03:50:45.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc410e04700 1 -- 172.21.15.79:0/665799647 learned_addr learned my addr 172.21.15.79:0/665799647 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.114+0000 7fc411e06700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0ceff0 0x7fc40c1f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0003d00 con 0x7f5dc00ad640 2023-06-08T03:50:45.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0003e60 con 0x7f5dc00ad640 2023-06-08T03:50:45.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db001d670 con 0x7f5dc00ad640 2023-06-08T03:50:45.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db0009430 con 0x7f5dc00ad640 2023-06-08T03:50:45.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc410e04700 1 -- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0ceff0 msgr2=0x7fc40c1f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc410e04700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0ceff0 0x7fc40c1f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc410e04700 1 -- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0713e0 msgr2=0x7fc40c0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc410e04700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0713e0 0x7fc40c0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc410e04700 1 -- 172.21.15.79:0/665799647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009430 con 0x7fc40c0cde80 2023-06-08T03:50:45.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc015d540 con 0x7f5dc00ad640 2023-06-08T03:50:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc411605700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0713e0 0x7fc40c0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc410e04700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 0x7fc40c0b37d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b4a0 tx=0x7fc3fc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f414f5a0 con 0x7f91f40b20d0 2023-06-08T03:50:45.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7f91e9ffb700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f91d806ace0 0x7f91d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7f91f9609700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f91d806ace0 0x7f91d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004230 con 0x7f5d500cef80 2023-06-08T03:50:45.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c004390 con 0x7f5d500cef80 2023-06-08T03:50:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7f5d4ffff700 1 -- 172.21.15.79:0/2312894821 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c010890 con 0x7f5d500cef80 2023-06-08T03:50:45.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f91f005cdb0 con 0x7f91f40b20d0 2023-06-08T03:50:45.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c011800 con 0x7f3978071fe0 2023-06-08T03:50:45.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c011960 con 0x7f3978071fe0 2023-06-08T03:50:45.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c011cd0 con 0x7f3978071fe0 2023-06-08T03:50:45.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7febbc0548c0 con 0x7febcc097a10 2023-06-08T03:50:45.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39780b3290 con 0x7f3978071fe0 2023-06-08T03:50:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f5d5739c700 1 -- 172.21.15.79:0/2312894821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500cef80 msgr2=0x7f5d5006c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2312894821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500cef80 0x7f5d5006c410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098d0 tx=0x7f5d3c009be0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcedc053860 con 0x7fced80043a0 2023-06-08T03:50:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f5d5739c700 1 -- 172.21.15.79:0/2312894821 shutdown_connections 2023-06-08T03:50:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2312894821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500cef80 0x7f5d5006c410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f5d5739c700 1 -- 172.21.15.79:0/2312894821 >> 172.21.15.79:0/2312894821 conn(0x7f5d50069d50 msgr2=0x7f5d5006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.115+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011830 con 0x7fc40c0cde80 2023-06-08T03:50:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc011990 con 0x7fc40c0cde80 2023-06-08T03:50:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.116+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc011d40 con 0x7fc40c0cde80 2023-06-08T03:50:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f5d5739c700 1 -- 172.21.15.79:0/2312894821 shutdown_connections 2023-06-08T03:50:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.118+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c06d2e0 con 0x7fc40c0cde80 2023-06-08T03:50:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39780b3850 con 0x7f3978071fe0 2023-06-08T03:50:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f414fbf0 con 0x7f91f40b20d0 2023-06-08T03:50:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.117+0000 7f91f9609700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f91d806ace0 0x7f91d806d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91e40016f0 tx=0x7f91e4007990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.118+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c151c70 con 0x7fc40c0cde80 2023-06-08T03:50:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.118+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5db001d7d0 con 0x7f5dc00ad640 2023-06-08T03:50:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.119+0000 7f5da7fff700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dac06ace0 0x7f5dac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.118+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978073e90 con 0x7f3978071fe0 2023-06-08T03:50:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.119+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5db008a9c0 con 0x7f5dc00ad640 2023-06-08T03:50:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.119+0000 7f5dbeffd700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dac06ace0 0x7f5dac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.118+0000 7f5d5739c700 1 -- 172.21.15.79:0/2312894821 wait complete. 2023-06-08T03:50:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5dbeffd700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dac06ace0 0x7f5dac06d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0202370 tx=0x7f5db40092c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 Processor -- start 2023-06-08T03:50:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 -- start start 2023-06-08T03:50:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 0x7f5d501413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d501581c0 0x7f5d500ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500ae9d0 0x7f5d500ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50141900 con 0x7f5d501581c0 2023-06-08T03:50:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500afb60 con 0x7f5d500704a0 2023-06-08T03:50:45.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.120+0000 7f5d5739c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500aef50 con 0x7f5d500ae9d0 2023-06-08T03:50:45.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 0x7f5d501413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d54937700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d501581c0 0x7f5d500ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 0x7f5d501413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58056/0 (socket says 172.21.15.79:58056) 2023-06-08T03:50:45.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 -- 172.21.15.79:0/2438878824 learned_addr learned my addr 172.21.15.79:0/2438878824 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 -- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500ae9d0 msgr2=0x7f5d500ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500ae9d0 0x7f5d500ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 -- 172.21.15.79:0/2438878824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d501581c0 msgr2=0x7f5d500ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d501581c0 0x7f5d500ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 -- 172.21.15.79:0/2438878824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0094a0 con 0x7f5d500704a0 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d54937700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d501581c0 0x7f5d500ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:45.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d55138700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 0x7f5d501413c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cc80 tx=0x7f5d4400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003c00 con 0x7f5d500704a0 2023-06-08T03:50:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44003d60 con 0x7f5d500704a0 2023-06-08T03:50:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.121+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400abb0 con 0x7f5d500704a0 2023-06-08T03:50:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.122+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d500af230 con 0x7f5d500704a0 2023-06-08T03:50:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.122+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d500af7f0 con 0x7f5d500704a0 2023-06-08T03:50:45.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.123+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d500af980 con 0x7f5d500704a0 2023-06-08T03:50:45.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.123+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d4400ad10 con 0x7f5d500704a0 2023-06-08T03:50:45.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.124+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91f008f060 con 0x7f91f40b20d0 2023-06-08T03:50:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.124+0000 7f5d4dffb700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5d3406ace0 0x7f5d3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.124+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5d44089990 con 0x7f5d500704a0 2023-06-08T03:50:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7f5d54937700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5d3406ace0 0x7f5d3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.126+0000 7f5d54937700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5d3406ace0 0x7f5d3406d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d500cf370 tx=0x7f5d3c014000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.129+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d440546b0 con 0x7f5d500704a0 2023-06-08T03:50:45.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.131+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc0073e90 con 0x7f5dc00ad640 2023-06-08T03:50:45.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3fc01a400 con 0x7fc40c0cde80 2023-06-08T03:50:45.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7fc401ffb700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc3f006ace0 0x7fc3f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc3fc089690 con 0x7fc40c0cde80 2023-06-08T03:50:45.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.126+0000 7fc411605700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc3f006ace0 0x7fc3f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.126+0000 7fc411605700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc3f006ace0 0x7fc3f006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc408009b40 tx=0x7fc408008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.131+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c151ef0 con 0x7fc40c0cde80 2023-06-08T03:50:45.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f396c010350 con 0x7f3978071fe0 2023-06-08T03:50:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7f395f7fe700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f396406ace0 0x7f396406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f396c0892e0 con 0x7f3978071fe0 2023-06-08T03:50:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.125+0000 7f39767fc700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f396406ace0 0x7f396406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.126+0000 7f39767fc700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f396406ace0 0x7f396406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3968000c00 tx=0x7f3968008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.133+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f396c0540b0 con 0x7f3978071fe0 2023-06-08T03:50:45.232 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.142+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc3fc054460 con 0x7fc40c0cde80 2023-06-08T03:50:45.233 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.146+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5db0055810 con 0x7f5dc00ad640 2023-06-08T03:50:45.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.152+0000 7fab0758b700 1 Processor -- start 2023-06-08T03:50:45.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.152+0000 7fab0758b700 1 -- start start 2023-06-08T03:50:45.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.153+0000 7fab0758b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.153+0000 7fab0758b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab000cb890 0x7fab000c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.153+0000 7fab0758b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab000713e0 0x7fab00071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.153+0000 7fab0758b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000ccbc0 con 0x7fab000cb890 2023-06-08T03:50:45.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.153+0000 7fab0758b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab000cc9c0 con 0x7fab000713e0 2023-06-08T03:50:45.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.153+0000 7fab0758b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab001609e0 con 0x7fab000c14b0 2023-06-08T03:50:45.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.154+0000 7fab04b26700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab000cb890 0x7fab000c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.154+0000 7fab04b26700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab000cb890 0x7fab000c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35178/0 (socket says 172.21.15.79:35178) 2023-06-08T03:50:45.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.154+0000 7fab04b26700 1 -- 172.21.15.79:0/1490359210 learned_addr learned my addr 172.21.15.79:0/1490359210 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.155+0000 7fab04b26700 1 -- 172.21.15.79:0/1490359210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 msgr2=0x7fab000ae300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-08T03:50:45.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.155+0000 7fab04b26700 1 --2- 172.21.15.79:0/1490359210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000ae300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.155+0000 7fab04b26700 1 -- 172.21.15.79:0/1490359210 >> v2:172.21.15.156:3300/0 conn(0x7fab000713e0 msgr2=0x7fab00071860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-08T03:50:45.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.155+0000 7fab04b26700 1 --2- 172.21.15.79:0/1490359210 >> v2:172.21.15.156:3300/0 conn(0x7fab000713e0 0x7fab00071860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.155+0000 7fab04b26700 1 -- 172.21.15.79:0/1490359210 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000c1dc0 con 0x7fab000cb890 2023-06-08T03:50:45.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.155+0000 7fab04b26700 1 --2- 172.21.15.79:0/1490359210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000cb890 0x7fab000c1880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab00070ef0 tx=0x7faaf0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50957009c3c3520 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.156+0000 7faaf7fff700 1 -- 172.21.15.79:0/1490359210 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000b870 con 0x7fab000cb890 2023-06-08T03:50:45.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.156+0000 7faaf7fff700 1 -- 172.21.15.79:0/1490359210 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf000b9d0 con 0x7fab000cb890 2023-06-08T03:50:45.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.156+0000 7faaf7fff700 1 -- 172.21.15.79:0/1490359210 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf000bd60 con 0x7fab000cb890 2023-06-08T03:50:45.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.157+0000 7fab0758b700 1 -- 172.21.15.79:0/1490359210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000cb890 msgr2=0x7fab000c1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.157+0000 7fab0758b700 1 --2- 172.21.15.79:0/1490359210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000cb890 0x7fab000c1880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab00070ef0 tx=0x7faaf0009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.158+0000 7fab0758b700 1 -- 172.21.15.79:0/1490359210 shutdown_connections 2023-06-08T03:50:45.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.158+0000 7fab0758b700 1 --2- 172.21.15.79:0/1490359210 >> v2:172.21.15.156:3300/0 conn(0x7fab000713e0 0x7fab00071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.158+0000 7fab0758b700 1 --2- 172.21.15.79:0/1490359210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.158+0000 7fab0758b700 1 --2- 172.21.15.79:0/1490359210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000cb890 0x7fab000c1880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.158+0000 7fab0758b700 1 -- 172.21.15.79:0/1490359210 >> 172.21.15.79:0/1490359210 conn(0x7fab000ce4e0 msgr2=0x7fab000ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.158+0000 7fab0758b700 1 -- 172.21.15.79:0/1490359210 shutdown_connections 2023-06-08T03:50:45.240 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.159+0000 7fab0758b700 1 -- 172.21.15.79:0/1490359210 wait complete. 2023-06-08T03:50:45.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.160+0000 7fab0758b700 1 Processor -- start 2023-06-08T03:50:45.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.161+0000 7fab0758b700 1 -- start start 2023-06-08T03:50:45.241 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.162+0000 7fab0758b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 0x7fab000b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.162+0000 7fab0758b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.162+0000 7fab0758b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab000cb890 0x7fab000963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.162+0000 7fab0758b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0014b2f0 con 0x7fab000713e0 2023-06-08T03:50:45.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.162+0000 7fab0758b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0014be30 con 0x7fab000cb890 2023-06-08T03:50:45.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.162+0000 7fab0758b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0014b500 con 0x7fab000c14b0 2023-06-08T03:50:45.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 0x7fab000b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab04b26700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 0x7fab000b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35182/0 (socket says 172.21.15.79:35182) 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 -- 172.21.15.79:0/3226233337 learned_addr learned my addr 172.21.15.79:0/3226233337 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 -- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 msgr2=0x7fab000b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 -- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab000cb890 msgr2=0x7fab000963a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab000cb890 0x7fab000963a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.163+0000 7fab05327700 1 -- 172.21.15.79:0/3226233337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf0009550 con 0x7fab000713e0 2023-06-08T03:50:45.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.164+0000 7fab05327700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 0x7fab000b34e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faaf800ba50 tx=0x7faaf800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.174+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf800cae0 con 0x7fab000713e0 2023-06-08T03:50:45.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.174+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf800cc40 con 0x7fab000713e0 2023-06-08T03:50:45.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.174+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8014440 con 0x7fab000713e0 2023-06-08T03:50:45.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.174+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0014b7e0 con 0x7fab000713e0 2023-06-08T03:50:45.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.174+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab000caf30 con 0x7fab000713e0 2023-06-08T03:50:45.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.176+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae00052f0 con 0x7fab000713e0 2023-06-08T03:50:45.245 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.187+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faaf8007370 con 0x7fab000713e0 2023-06-08T03:50:45.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.187+0000 7faaf5ffb700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faae406ad80 0x7faae406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:45.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.188+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faaf8089a50 con 0x7fab000713e0 2023-06-08T03:50:45.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.188+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faaf80b53b0 con 0x7fab000713e0 2023-06-08T03:50:45.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.196+0000 7fab04b26700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faae406ad80 0x7faae406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:45.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.246+0000 7fab04b26700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faae406ad80 0x7faae406d230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab0014fbb0 tx=0x7faaf0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:45.519 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-08T03:50:45.520 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.516+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5d50070b70 con 0x7f5d500704a0 2023-06-08T03:50:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.515+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fced40055a0 con 0x7fced80043a0 2023-06-08T03:50:45.521 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.515+0000 7fcee2ffd700 1 -- 172.21.15.79:0/2169914430 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcedc056e80 con 0x7fced80043a0 2023-06-08T03:50:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.517+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7febcc0ce070 con 0x7febcc097a10 2023-06-08T03:50:45.522 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.517+0000 7f5d4dffb700 1 -- 172.21.15.79:0/2438878824 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d44057cd0 con 0x7f5d500704a0 2023-06-08T03:50:45.523 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-08T03:50:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.518+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fced006ace0 msgr2=0x7fced006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.518+0000 7fcef4781700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fced006ace0 0x7fced006d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fceec1f9440 tx=0x7fcee4009510 comp rx=0 tx=0).stop 2023-06-08T03:50:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.518+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 msgr2=0x7fceec15ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.518+0000 7fcef4781700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 0x7fceec15ad70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c460 tx=0x7fcedc00c770 comp rx=0 tx=0).stop 2023-06-08T03:50:45.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.518+0000 7febb37fe700 1 -- 172.21.15.79:0/484643521 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febbc057ee0 con 0x7febcc097a10 2023-06-08T03:50:45.526 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:45.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.518+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f91f4073e90 con 0x7f91f40b20d0 2023-06-08T03:50:45.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.519+0000 7f91e9ffb700 1 -- 172.21.15.79:0/2894075846 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91f0055130 con 0x7f91f40b20d0 2023-06-08T03:50:45.527 INFO:teuthology.orchestra.run.smithi079.stdout:55834574854 2023-06-08T03:50:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 shutdown_connections 2023-06-08T03:50:45.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7fcef4781700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fceec0afeb0 0x7fceec0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7fcef4781700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fced006ace0 0x7fced006d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7fcef4781700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fced80043a0 0x7fceec15ad70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7fcef4781700 1 --2- 172.21.15.79:0/2169914430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fceec15a7e0 0x7fceec08a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.529 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 >> 172.21.15.79:0/2169914430 conn(0x7fceec0ce4e0 msgr2=0x7fceec0cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5d3406ace0 msgr2=0x7f5d3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.530 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5d3406ace0 0x7f5d3406d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d500cf370 tx=0x7f5d3c014000 comp rx=0 tx=0).stop 2023-06-08T03:50:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 msgr2=0x7f5d501413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 0x7f5d501413c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cc80 tx=0x7f5d4400cf90 comp rx=0 tx=0).stop 2023-06-08T03:50:45.531 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 shutdown_connections 2023-06-08T03:50:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5d3406ace0 0x7f5d3406d190 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d500cf370 tx=0x7f5d3c014000 comp rx=0 tx=0).stop 2023-06-08T03:50:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d500ae9d0 0x7f5d500ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d500704a0 0x7f5d501413c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 --2- 172.21.15.79:0/2438878824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d501581c0 0x7f5d500ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 >> 172.21.15.79:0/2438878824 conn(0x7f5d50069d50 msgr2=0x7f5d5006c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f91d806ace0 msgr2=0x7f91d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f91fb86d700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f91d806ace0 0x7f91d806d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91e40016f0 tx=0x7f91e4007990 comp rx=0 tx=0).stop 2023-06-08T03:50:45.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 msgr2=0x7f91f406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.534 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f91fb86d700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 0x7f91f406c660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f91f406e1d0 tx=0x7f91f0004750 comp rx=0 tx=0).stop 2023-06-08T03:50:45.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 shutdown_connections 2023-06-08T03:50:45.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f5d5739c700 1 -- 172.21.15.79:0/2438878824 wait complete. 2023-06-08T03:50:45.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.521+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 shutdown_connections 2023-06-08T03:50:45.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91f40c9f50 0x7f91f40c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f91d806ace0 0x7f91d806d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f91f40af340 0x7f91f406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 --2- 172.21.15.79:0/2894075846 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f91f40b20d0 0x7f91f406c660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 >> 172.21.15.79:0/2894075846 conn(0x7f91f40ce4e0 msgr2=0x7f91f40af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 shutdown_connections 2023-06-08T03:50:45.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7fcef4781700 1 -- 172.21.15.79:0/2169914430 wait complete. 2023-06-08T03:50:45.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 shutdown_connections 2023-06-08T03:50:45.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.522+0000 7f91fb86d700 1 -- 172.21.15.79:0/2894075846 wait complete. 2023-06-08T03:50:45.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7febb806ad30 msgr2=0x7febb806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7febb806ad30 0x7febb806d1e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc0011000 comp rx=0 tx=0).stop 2023-06-08T03:50:45.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 msgr2=0x7febcc206b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 0x7febcc206b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febcc0b4550 tx=0x7febbc00b6c0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 shutdown_connections 2023-06-08T03:50:45.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febcc0cd790 0x7febcc1f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7febb806ad30 0x7febb806d1e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febcc0963b0 0x7febcc0c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 --2- 172.21.15.79:0/484643521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febcc097a10 0x7febcc206b20 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febcc0b4550 tx=0x7febbc00b6c0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.524+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 >> 172.21.15.79:0/484643521 conn(0x7febcc0ce4e0 msgr2=0x7febcc0c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.525+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 shutdown_connections 2023-06-08T03:50:45.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.525+0000 7febd1546700 1 -- 172.21.15.79:0/484643521 wait complete. 2023-06-08T03:50:45.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.537+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5dc00b4340 con 0x7f5dc00ad640 2023-06-08T03:50:45.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.537+0000 7f5da7fff700 1 -- 172.21.15.79:0/3202963408 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5db0016020 con 0x7f5dc00ad640 2023-06-08T03:50:45.544 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:45.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.538+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f397806ed70 con 0x7f3978071fe0 2023-06-08T03:50:45.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-06-08T03:50:45.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.540+0000 7f395f7fe700 1 -- 172.21.15.79:0/2112475037 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f396c0576d0 con 0x7f3978071fe0 2023-06-08T03:50:45.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.540+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc40c073e90 con 0x7fc40c0cde80 2023-06-08T03:50:45.548 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:45.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7fc401ffb700 1 -- 172.21.15.79:0/665799647 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3fc015070 con 0x7fc40c0cde80 2023-06-08T03:50:45.549 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:45.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dac06ace0 msgr2=0x7f5dac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dac06ace0 0x7f5dac06d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0202370 tx=0x7f5db40092c0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 msgr2=0x7f5dc00c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 0x7f5dc00c6360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5db0000c00 tx=0x7f5db0004750 comp rx=0 tx=0).stop 2023-06-08T03:50:45.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 shutdown_connections 2023-06-08T03:50:45.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dac06ace0 0x7f5dac06d190 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0202370 tx=0x7f5db40092c0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc015c450 0x7f5dc00c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc00ad640 0x7f5dc00c6360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 --2- 172.21.15.79:0/3202963408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc00c8b30 0x7f5dc00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.541+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 >> 172.21.15.79:0/3202963408 conn(0x7f5dc00ce4e0 msgr2=0x7f5dc015b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.542+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 shutdown_connections 2023-06-08T03:50:45.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.542+0000 7f5dc61ef700 1 -- 172.21.15.79:0/3202963408 wait complete. 2023-06-08T03:50:45.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc3f006ace0 msgr2=0x7fc3f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc3f006ace0 0x7fc3f006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc408009b40 tx=0x7fc408008070 comp rx=0 tx=0).stop 2023-06-08T03:50:45.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 msgr2=0x7fc40c0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 0x7fc40c0b37d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b4a0 tx=0x7fc3fc00b7b0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f396406ace0 msgr2=0x7f396406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f396406ace0 0x7f396406d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3968000c00 tx=0x7f3968008030 comp rx=0 tx=0).stop 2023-06-08T03:50:45.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 msgr2=0x7f3978152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 0x7f3978152c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f396c00c7f0 tx=0x7f396c00cb00 comp rx=0 tx=0).stop 2023-06-08T03:50:45.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 shutdown_connections 2023-06-08T03:50:45.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc3f006ace0 0x7fc3f006d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc40c0ceff0 0x7fc40c1f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc40c0713e0 0x7fc40c0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 --2- 172.21.15.79:0/665799647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc40c0cde80 0x7fc40c0b37d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 shutdown_connections 2023-06-08T03:50:45.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f397815c1f0 0x7f3978155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3978071fe0 0x7f3978152c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f396406ace0 0x7f396406d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 --2- 172.21.15.79:0/2112475037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f39780cf130 0x7f3978153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 >> 172.21.15.79:0/2112475037 conn(0x7f39780b1d30 msgr2=0x7f39780cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 >> 172.21.15.79:0/665799647 conn(0x7fc40c0ce4e0 msgr2=0x7fc40c0ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 shutdown_connections 2023-06-08T03:50:45.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 shutdown_connections 2023-06-08T03:50:45.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7f397d6e4700 1 -- 172.21.15.79:0/2112475037 wait complete. 2023-06-08T03:50:45.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.545+0000 7fc413869700 1 -- 172.21.15.79:0/665799647 wait complete. 2023-06-08T03:50:45.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-06-08T03:50:45.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-06-08T03:50:45.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2023-06-08T03:50:45.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-06-08T03:50:45.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-06-08T03:50:45.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-06-08T03:50:45.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.569+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7faae0005870 con 0x7fab000713e0 2023-06-08T03:50:45.573 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.569+0000 7faaf5ffb700 1 -- 172.21.15.79:0/3226233337 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faaf80547f0 con 0x7fab000713e0 2023-06-08T03:50:45.573 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:45.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faae406ad80 msgr2=0x7faae406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faae406ad80 0x7faae406d230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab0014fbb0 tx=0x7faaf0008030 comp rx=0 tx=0).stop 2023-06-08T03:50:45.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 msgr2=0x7fab000b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:45.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 0x7fab000b34e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faaf800ba50 tx=0x7faaf800be10 comp rx=0 tx=0).stop 2023-06-08T03:50:45.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 shutdown_connections 2023-06-08T03:50:45.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab000cb890 0x7fab000963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab000713e0 0x7fab000b34e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faae406ad80 0x7faae406d230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 --2- 172.21.15.79:0/3226233337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab000c14b0 0x7fab000b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:45.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 >> 172.21.15.79:0/3226233337 conn(0x7fab000ce4e0 msgr2=0x7fab000ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:45.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 shutdown_connections 2023-06-08T03:50:45.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:45.572+0000 7fab0758b700 1 -- 172.21.15.79:0/3226233337 wait complete. 2023-06-08T03:50:45.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-06-08T03:50:46.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-08T03:50:46.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-08T03:50:46.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-08T03:50:46.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-08T03:50:46.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-08T03:50:46.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-08T03:50:46.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-08T03:50:46.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-08T03:50:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.767+0000 7fa737f33700 1 Processor -- start 2023-06-08T03:50:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.767+0000 7fa737f33700 1 -- start start 2023-06-08T03:50:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.768+0000 7fa737f33700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa7300cb2f0 0x7fa7300707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.768+0000 7fa737f33700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73006d420 con 0x7fa730164640 2023-06-08T03:50:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.768+0000 7fa737f33700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa730070d10 con 0x7fa7300cb6c0 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.768+0000 7fa737f33700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7300cdda0 con 0x7fa7300cb2f0 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.769+0000 7fa7354ce700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fa7300cb6c0 0x7fa730163f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60826/0 (socket says 172.21.15.79:60826) 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.769+0000 7fa7354ce700 1 -- 172.21.15.79:0/442506098 learned_addr learned my addr 172.21.15.79:0/442506098 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167432992 0 0) 0x7fa73006d420 con 0x7fa730164640 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa71c0035b0 con 0x7fa730164640 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa735ccf700 1 --2- 172.21.15.79:0/442506098 >> v2:172.21.15.156:3300/0 conn(0x7fa7300cb2f0 0x7fa7300707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781171970 0 0) 0x7fa71c0035b0 con 0x7fa730164640 2023-06-08T03:50:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 >> v2:172.21.15.156:3300/0 conn(0x7fa7300cb2f0 msgr2=0x7fa7300707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 --2- 172.21.15.79:0/442506098 >> v2:172.21.15.156:3300/0 conn(0x7fa7300cb2f0 0x7fa7300707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 >> v1:172.21.15.156:6789/0 conn(0x7fa7300cb6c0 legacy=0x7fa730163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.770+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73015ad50 con 0x7fa730164640 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fa72c001670 con 0x7fa730164640 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 >> v1:172.21.15.79:6789/0 conn(0x7fa730164640 legacy=0x7fa73016b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa71c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c004850 0x7fa71c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c005200 0x7fa71c02da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c02e0f0 con 0x7fa71c003a00 2023-06-08T03:50:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c02e260 con 0x7fa71c004850 2023-06-08T03:50:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.771+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c02e420 con 0x7fa71c005200 2023-06-08T03:50:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.776+0000 7fa735ccf700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa71c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa735ccf700 1 -- 172.21.15.79:0/442506098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c005200 msgr2=0x7fa71c02da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa735ccf700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa71c005200 0x7fa71c02da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa735ccf700 1 -- 172.21.15.79:0/442506098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c004850 msgr2=0x7fa71c004cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa735ccf700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71c004850 0x7fa71c004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa735ccf700 1 -- 172.21.15.79:0/442506098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73015ad50 con 0x7fa71c003a00 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa735ccf700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa71c003df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa720009880 tx=0x7fa71c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0172cd1c2a52c85 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa7364d0700 1 -- 172.21.15.79:0/442506098 reap_dead start 2023-06-08T03:50:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000c070 con 0x7fa71c003a00 2023-06-08T03:50:46.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720013610 con 0x7fa71c003a00 2023-06-08T03:50:46.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.777+0000 7fa734ccd700 1 -- 172.21.15.79:0/442506098 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720013980 con 0x7fa71c003a00 2023-06-08T03:50:46.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.778+0000 7fa737f33700 1 -- 172.21.15.79:0/442506098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 msgr2=0x7fa71c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.778+0000 7fa737f33700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa71c003df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa720009880 tx=0x7fa71c005880 comp rx=0 tx=0).stop 2023-06-08T03:50:46.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.780+0000 7fa737f33700 1 -- 172.21.15.79:0/442506098 shutdown_connections 2023-06-08T03:50:46.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.780+0000 7fa737f33700 1 --2- 172.21.15.79:0/442506098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa71c003df0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.780+0000 7fa737f33700 1 -- 172.21.15.79:0/442506098 >> 172.21.15.79:0/442506098 conn(0x7fa730069d50 msgr2=0x7fa73006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.788+0000 7fa737f33700 1 -- 172.21.15.79:0/442506098 shutdown_connections 2023-06-08T03:50:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.789+0000 7fa737f33700 1 -- 172.21.15.79:0/442506098 wait complete. 2023-06-08T03:50:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.790+0000 7fa737f33700 1 Processor -- start 2023-06-08T03:50:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.790+0000 7fa737f33700 1 -- start start 2023-06-08T03:50:46.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.791+0000 7fa737f33700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa7301fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa737f33700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7300ae500 0x7fa7300ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa737f33700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73015faf0 0x7fa73015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa737f33700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301604d0 con 0x7fa71c003a00 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa737f33700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300aef50 con 0x7fa73015faf0 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa737f33700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300af230 con 0x7fa7300ae500 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa7301fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa7301fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35200/0 (socket says 172.21.15.79:35200) 2023-06-08T03:50:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 -- 172.21.15.79:0/470137892 learned_addr learned my addr 172.21.15.79:0/470137892 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa7364d0700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73015faf0 0x7fa73015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa7354ce700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7300ae500 0x7fa7300ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 -- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7300ae500 msgr2=0x7fa7300ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7300ae500 0x7fa7300ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 -- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73015faf0 msgr2=0x7fa73015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73015faf0 0x7fa73015ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.792+0000 7fa735ccf700 1 -- 172.21.15.79:0/470137892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300af510 con 0x7fa71c003a00 2023-06-08T03:50:46.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.793+0000 7fa735ccf700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa7301fe100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa71c02e230 tx=0x7fa7200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.793+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720013410 con 0x7fa71c003a00 2023-06-08T03:50:46.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.793+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72001c630 con 0x7fa71c003a00 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.793+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72001c9e0 con 0x7fa71c003a00 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.793+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7300af7f0 con 0x7fa71c003a00 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.793+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7301fe640 con 0x7fa71c003a00 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.795+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa72001cb40 con 0x7fa71c003a00 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.795+0000 7fa7267fc700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa71406ace0 0x7fa71406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.796+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa72008a030 con 0x7fa71c003a00 2023-06-08T03:50:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.796+0000 7fa7354ce700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa71406ace0 0x7fa71406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.796+0000 7f174a7cd700 1 Processor -- start 2023-06-08T03:50:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.796+0000 7f174a7cd700 1 -- start start 2023-06-08T03:50:46.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.797+0000 7fa7354ce700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa71406ace0 0x7fa71406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa7300b0010 tx=0x7fa728009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.797+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7180052f0 con 0x7fa71c003a00 2023-06-08T03:50:46.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f174a7cd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1744159f70 0x7f17440ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f174a7cd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f174a7cd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f17440af160 0x7f17440af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f174a7cd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440afb40 con 0x7f17440af160 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f174a7cd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440aed60 con 0x7f1744159f70 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f174a7cd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17440c9de0 con 0x7f17440ad830 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49532/0 (socket says 172.21.15.79:49532) 2023-06-08T03:50:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 -- 172.21.15.79:0/1229810664 learned_addr learned my addr 172.21.15.79:0/1229810664 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f1743fff700 1 --2- 172.21.15.79:0/1229810664 >> v2:172.21.15.156:3300/0 conn(0x7f1744159f70 0x7f17440ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 -- 172.21.15.79:0/1229810664 >> v2:172.21.15.156:3300/0 conn(0x7f1744159f70 msgr2=0x7f17440ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 --2- 172.21.15.79:0/1229810664 >> v2:172.21.15.156:3300/0 conn(0x7f1744159f70 0x7f17440ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 -- 172.21.15.79:0/1229810664 >> v2:172.21.15.79:3300/0 conn(0x7f17440af160 msgr2=0x7f17440af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 --2- 172.21.15.79:0/1229810664 >> v2:172.21.15.79:3300/0 conn(0x7f17440af160 0x7f17440af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7f17437fe700 1 -- 172.21.15.79:0/1229810664 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440ca080 con 0x7f17440ad830 2023-06-08T03:50:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.798+0000 7ff2cfed4700 1 Processor -- start 2023-06-08T03:50:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.799+0000 7f17437fe700 1 --2- 172.21.15.79:0/1229810664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440ae820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17440ce370 tx=0x7f1734009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318faf955a62f7da server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.799+0000 7f1742ffd700 1 -- 172.21.15.79:0/1229810664 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734011830 con 0x7f17440ad830 2023-06-08T03:50:46.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.799+0000 7f1742ffd700 1 -- 172.21.15.79:0/1229810664 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1734011990 con 0x7f17440ad830 2023-06-08T03:50:46.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7f174a7cd700 1 -- 172.21.15.79:0/1229810664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 msgr2=0x7f17440ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7f174a7cd700 1 --2- 172.21.15.79:0/1229810664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440ae820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17440ce370 tx=0x7f1734009bd0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.799+0000 7ff2cfed4700 1 -- start start 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cfed4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2c8158780 0x7ff2c806b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cfed4700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80ccb30 0x7ff2c80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cfed4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c806da90 con 0x7ff2c80707d0 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cfed4700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c806e110 con 0x7ff2c8158780 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cfed4700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80ce690 con 0x7ff2c80ccb30 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cd46f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80ccb30 0x7ff2c80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cd46f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80ccb30 0x7ff2c80c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49536/0 (socket says 172.21.15.79:49536) 2023-06-08T03:50:46.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cd46f700 1 -- 172.21.15.79:0/815452704 learned_addr learned my addr 172.21.15.79:0/815452704 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7ff2cdc70700 1 --2- 172.21.15.79:0/815452704 >> v2:172.21.15.156:3300/0 conn(0x7ff2c8158780 0x7ff2c806b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7fe44f783700 1 Processor -- start 2023-06-08T03:50:46.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.800+0000 7fe44f783700 1 -- start start 2023-06-08T03:50:46.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7ff2cdc70700 1 -- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80ccb30 msgr2=0x7ff2c80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7ff2cdc70700 1 --2- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80ccb30 0x7ff2c80c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7ff2cdc70700 1 -- 172.21.15.79:0/815452704 >> v1:172.21.15.156:6789/0 conn(0x7ff2c80707d0 legacy=0x7ff2c806cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7ff2cdc70700 1 -- 172.21.15.79:0/815452704 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c80ce970 con 0x7ff2c8158780 2023-06-08T03:50:46.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 -- 172.21.15.79:0/1229810664 shutdown_connections 2023-06-08T03:50:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 --2- 172.21.15.79:0/1229810664 >> v2:172.21.15.79:3300/0 conn(0x7f17440af160 0x7f17440af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 --2- 172.21.15.79:0/1229810664 >> v2:172.21.15.156:3300/0 conn(0x7f1744159f70 0x7f17440ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 --2- 172.21.15.79:0/1229810664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440ae820 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 -- 172.21.15.79:0/1229810664 >> 172.21.15.79:0/1229810664 conn(0x7f17440ce4e0 msgr2=0x7f17440ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7ff2cdc70700 1 --2- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c806b230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009840 tx=0x7ff2b8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb3dd41aa377eb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 -- 172.21.15.79:0/1229810664 shutdown_connections 2023-06-08T03:50:46.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7fe44f783700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480ae300 0x7fe4480cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7fe44f783700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4480ccfe0 0x7fe4480cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7fe44f783700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4480cc9c0 con 0x7fe4480c14b0 2023-06-08T03:50:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7fe44f783700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4480ccbc0 con 0x7fe4480ccfe0 2023-06-08T03:50:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7fe44f783700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480ae770 con 0x7fe4480ae300 2023-06-08T03:50:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4480ccfe0 0x7fe4480cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4480ccfe0 0x7fe4480cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58114/0 (socket says 172.21.15.79:58114) 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 -- 172.21.15.79:0/53737211 learned_addr learned my addr 172.21.15.79:0/53737211 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 -- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480ae300 msgr2=0x7fe4480cb890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 --2- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480ae300 0x7fe4480cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 -- 172.21.15.79:0/53737211 >> v1:172.21.15.156:6789/0 conn(0x7fe4480c14b0 legacy=0x7fe4480c1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 -- 172.21.15.79:0/53737211 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4481609e0 con 0x7fe4480ccfe0 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe44dd20700 1 --2- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480cd460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe440009e80 tx=0x7fe44000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff037c80ffbbd801 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe43ffff700 1 -- 172.21.15.79:0/53737211 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000e070 con 0x7fe4480ccfe0 2023-06-08T03:50:46.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe43ffff700 1 -- 172.21.15.79:0/53737211 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440003a20 con 0x7fe4480ccfe0 2023-06-08T03:50:46.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.802+0000 7fe43ffff700 1 -- 172.21.15.79:0/53737211 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440003dd0 con 0x7fe4480ccfe0 2023-06-08T03:50:46.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.803+0000 7fe44f783700 1 -- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 msgr2=0x7fe4480cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.803+0000 7fe44f783700 1 --2- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480cd460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe440009e80 tx=0x7fe44000b330 comp rx=0 tx=0).stop 2023-06-08T03:50:46.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.801+0000 7f174a7cd700 1 -- 172.21.15.79:0/1229810664 wait complete. 2023-06-08T03:50:46.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.803+0000 7f174a7cd700 1 Processor -- start 2023-06-08T03:50:46.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.804+0000 7ff2ccc6e700 1 -- 172.21.15.79:0/815452704 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8004230 con 0x7ff2c8158780 2023-06-08T03:50:46.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.804+0000 7ff2ccc6e700 1 -- 172.21.15.79:0/815452704 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b8004390 con 0x7ff2c8158780 2023-06-08T03:50:46.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.804+0000 7ff2ccc6e700 1 -- 172.21.15.79:0/815452704 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80036a0 con 0x7ff2c8158780 2023-06-08T03:50:46.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.804+0000 7f174a7cd700 1 -- start start 2023-06-08T03:50:46.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7ff2cfed4700 1 -- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 msgr2=0x7ff2c806b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c806b230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009840 tx=0x7ff2b8009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:46.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7fe44f783700 1 -- 172.21.15.79:0/53737211 shutdown_connections 2023-06-08T03:50:46.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7fe44f783700 1 --2- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480cd460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7fe44f783700 1 --2- 172.21.15.79:0/53737211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480ae300 0x7fe4480cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7fe44f783700 1 -- 172.21.15.79:0/53737211 >> 172.21.15.79:0/53737211 conn(0x7fe4480ce4e0 msgr2=0x7fe4480ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa720054e80 con 0x7fa71c003a00 2023-06-08T03:50:46.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.808+0000 7f3e96b4c700 1 Processor -- start 2023-06-08T03:50:46.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.809+0000 7f3e96b4c700 1 -- start start 2023-06-08T03:50:46.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e96b4c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3e900cc690 0x7f3e90163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e96b4c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e900cd7b0 con 0x7f3e900ca360 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e96b4c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e900cda50 con 0x7f3e90158d50 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e96b4c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900cdd30 con 0x7f3e900cc690 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e948e8700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3e90158d50 0x7f3e9015d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60866/0 (socket says 172.21.15.79:60866) 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e948e8700 1 -- 172.21.15.79:0/3892903462 learned_addr learned my addr 172.21.15.79:0/3892903462 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945301611 0 0) 0x7f3e900cda50 con 0x7f3e90158d50 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e7c0035b0 con 0x7f3e90158d50 2023-06-08T03:50:46.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509439213 0 0) 0x7f3e7c0035b0 con 0x7f3e90158d50 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 >> v2:172.21.15.79:3300/0 conn(0x7f3e900cc690 msgr2=0x7f3e90163d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e8f7fe700 1 --2- 172.21.15.79:0/3892903462 >> v2:172.21.15.79:3300/0 conn(0x7f3e900cc690 0x7f3e90163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 >> v1:172.21.15.79:6789/0 conn(0x7f3e900ca360 legacy=0x7f3e900cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900ce010 con 0x7f3e90158d50 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f3e88001a30 con 0x7f3e90158d50 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 >> v1:172.21.15.156:6789/0 conn(0x7f3e90158d50 legacy=0x7f3e9015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e7c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e7c004850 0x7f3e7c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c005320 0x7f3e7c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900cdd30 con 0x7f3e7c004850 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c00c1e0 con 0x7f3e7c005320 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c00c3a0 con 0x7f3e7c003a00 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e950e9700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c005320 0x7f3e7c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e7c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 -- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c005320 msgr2=0x7f3e7c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e7c005320 0x7f3e7c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 -- 172.21.15.79:0/3892903462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e7c004850 msgr2=0x7f3e7c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e7c004850 0x7f3e7c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 -- 172.21.15.79:0/3892903462 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0035b0 con 0x7f3e7c003a00 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e950e9700 1 -- 172.21.15.79:0/3892903462 reap_dead start 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7f3e948e8700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e7c003df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800a840 tx=0x7f3e7c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d468f19e19fd1c0a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88012600 con 0x7f3e7c003a00 2023-06-08T03:50:46.823 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e88012760 con 0x7f3e7c003a00 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e8f7fe700 1 -- 172.21.15.79:0/3892903462 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e88012b10 con 0x7f3e7c003a00 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 -- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 msgr2=0x7f3e7c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e7c003df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800a840 tx=0x7f3e7c005880 comp rx=0 tx=0).stop 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 -- 172.21.15.79:0/3892903462 shutdown_connections 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/3892903462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e7c003df0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 -- 172.21.15.79:0/3892903462 >> 172.21.15.79:0/3892903462 conn(0x7f3e900b1d30 msgr2=0x7f3e9006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 -- 172.21.15.79:0/3892903462 shutdown_connections 2023-06-08T03:50:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7f3e96b4c700 1 -- 172.21.15.79:0/3892903462 wait complete. 2023-06-08T03:50:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f3e96b4c700 1 Processor -- start 2023-06-08T03:50:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f3e96b4c700 1 -- start start 2023-06-08T03:50:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f174a7cd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f174a7cd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17440af160 0x7f17441fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f174a7cd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744159f70 0x7f17441fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f174a7cd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17440b3d00 con 0x7f1744159f70 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f174a7cd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174415b390 con 0x7f17440af160 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f174a7cd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17441f1da0 con 0x7f17440ad830 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f1743fff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f1743fff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49544/0 (socket says 172.21.15.79:49544) 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.805+0000 7f1743fff700 1 -- 172.21.15.79:0/2646947848 learned_addr learned my addr 172.21.15.79:0/2646947848 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1743fff700 1 -- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17440af160 msgr2=0x7f17441fe2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1743fff700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17440af160 0x7f17441fe2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1743fff700 1 -- 172.21.15.79:0/2646947848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744159f70 msgr2=0x7f17441fe7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1743fff700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744159f70 0x7f17441fe7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1743fff700 1 -- 172.21.15.79:0/2646947848 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734009550 con 0x7f17440ad830 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1743fff700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440b2200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f173c00eed0 tx=0x7f173c00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c010040 con 0x7f17440ad830 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c00bdb0 con 0x7f17440ad830 2023-06-08T03:50:46.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c014620 con 0x7f17440ad830 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17440b2ac0 con 0x7f17440ad830 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174415b570 con 0x7f17440ad830 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.808+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f173c014780 con 0x7f17440ad830 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.809+0000 7f1740ff9700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f173006ad30 0x7f173006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.809+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f173c089bf0 con 0x7f17440ad830 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.809+0000 7f17437fe700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f173006ad30 0x7f173006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7f17437fe700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f173006ad30 0x7f173006d1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17440ce370 tx=0x7f1734008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.813+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174415b8e0 con 0x7f17440ad830 2023-06-08T03:50:46.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7ff2cfed4700 1 -- 172.21.15.79:0/815452704 shutdown_connections 2023-06-08T03:50:46.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c806b230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/815452704 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80ccb30 0x7ff2c80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7ff2cfed4700 1 -- 172.21.15.79:0/815452704 >> 172.21.15.79:0/815452704 conn(0x7ff2c80ced30 msgr2=0x7ff2c80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.813+0000 7ff2cfed4700 1 -- 172.21.15.79:0/815452704 shutdown_connections 2023-06-08T03:50:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.813+0000 7ff2cfed4700 1 -- 172.21.15.79:0/815452704 wait complete. 2023-06-08T03:50:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7ff2cfed4700 1 Processor -- start 2023-06-08T03:50:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7ff2cfed4700 1 -- start start 2023-06-08T03:50:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.813+0000 7f7881636700 1 Processor -- start 2023-06-08T03:50:46.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.813+0000 7f7881636700 1 -- start start 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f7881636700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f787c06a460 0x7f787c06af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f7881636700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c06b630 con 0x7f787c0ce530 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f7881636700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c070900 con 0x7f787c160820 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f7881636700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c070be0 con 0x7f787c06a460 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f787affd700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f787c160820 0x7f787c069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60880/0 (socket says 172.21.15.79:60880) 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f787affd700 1 -- 172.21.15.79:0/747071189 learned_addr learned my addr 172.21.15.79:0/747071189 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:46.831 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661234511 0 0) 0x7f787c070900 con 0x7f787c160820 2023-06-08T03:50:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78680035b0 con 0x7f787c160820 2023-06-08T03:50:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168941612 0 0) 0x7f78680035b0 con 0x7f787c160820 2023-06-08T03:50:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 >> v2:172.21.15.79:3300/0 conn(0x7f787c06a460 msgr2=0x7f787c06af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7fe44f783700 1 -- 172.21.15.79:0/53737211 shutdown_connections 2023-06-08T03:50:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.806+0000 7fe44f783700 1 -- 172.21.15.79:0/53737211 wait complete. 2023-06-08T03:50:46.832 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.808+0000 7fe44f783700 1 Processor -- start 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 -- start start 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4480ae300 0x7fe4481621a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480c14b0 0x7fe4480b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448161120 con 0x7fe4480ae300 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448161400 con 0x7fe4480ccfe0 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.810+0000 7fe44f783700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481616e0 con 0x7fe4480c14b0 2023-06-08T03:50:46.833 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58128/0 (socket says 172.21.15.79:58128) 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 -- 172.21.15.79:0/2261392571 learned_addr learned my addr 172.21.15.79:0/2261392571 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 -- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480c14b0 msgr2=0x7fe4480b3380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480c14b0 0x7fe4480b3380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 -- 172.21.15.79:0/2261392571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4480ae300 msgr2=0x7fe4481621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4480ae300 0x7fe4481621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 -- 172.21.15.79:0/2261392571 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe440009b30 con 0x7fe4480ccfe0 2023-06-08T03:50:46.834 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe44dd20700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480c2d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe440015040 tx=0x7fe440004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.811+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440004500 con 0x7fe4480ccfe0 2023-06-08T03:50:46.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440004660 con 0x7fe4480ccfe0 2023-06-08T03:50:46.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440012430 con 0x7fe4480ccfe0 2023-06-08T03:50:46.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4481619c0 con 0x7fe4480ccfe0 2023-06-08T03:50:46.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.812+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe448158840 con 0x7fe4480ccfe0 2023-06-08T03:50:46.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe448073e90 con 0x7fe4480ccfe0 2023-06-08T03:50:46.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f8418aef700 1 Processor -- start 2023-06-08T03:50:46.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.814+0000 7f8418aef700 1 -- start start 2023-06-08T03:50:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f787a7fc700 1 --2- 172.21.15.79:0/747071189 >> v2:172.21.15.79:3300/0 conn(0x7f787c06a460 0x7f787c06af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 --2- 172.21.15.79:0/747071189 >> v2:172.21.15.79:3300/0 conn(0x7f787c06a460 0x7f787c06af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 >> v1:172.21.15.79:6789/0 conn(0x7f787c0ce530 legacy=0x7f787c0c1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c070ec0 con 0x7f787c160820 2023-06-08T03:50:46.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f78640016b0 con 0x7f787c160820 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 >> v1:172.21.15.156:6789/0 conn(0x7f787c160820 legacy=0x7f787c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f7868003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7868004850 0x7f7868004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7868005320 0x7f786800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c070be0 con 0x7f7868005320 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786800c210 con 0x7f7868003a00 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786800c3d0 con 0x7f7868004850 2023-06-08T03:50:46.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f787a7fc700 1 --2- 172.21.15.79:0/747071189 >> v2:172.21.15.79:3300/0 conn(0x7f787c06a460 0x7f787c06af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f3e96b4c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e90155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.815+0000 7f3e96b4c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 0x7f3e900c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e96b4c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e901f98d0 0x7f3e901f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e96b4c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c7140 con 0x7f3e901f98d0 2023-06-08T03:50:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e96b4c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9015a040 con 0x7f3e900b4000 2023-06-08T03:50:46.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e96b4c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900b4a00 con 0x7f3e7c003a00 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 0x7f3e900c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 0x7f3e900c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58144/0 (socket says 172.21.15.79:58144) 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 -- 172.21.15.79:0/2983784726 learned_addr learned my addr 172.21.15.79:0/2983784726 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e948e8700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e90155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e950e9700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e901f98d0 0x7f3e901f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 -- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 msgr2=0x7f3e90155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e90155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 -- 172.21.15.79:0/2983784726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e901f98d0 msgr2=0x7f3e901f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e901f98d0 0x7f3e901f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f3e8ffff700 1 -- 172.21.15.79:0/2983784726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900b4ce0 con 0x7f3e900b4000 2023-06-08T03:50:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f3e8ffff700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 0x7f3e900c6c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c4c0 tx=0x7f3e7800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cfed4700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80707d0 0x7ff2c80c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cfed4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2c80ccb30 0x7ff2c81e2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cfed4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c8153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cfed4700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b6810 con 0x7ff2c80ccb30 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cfed4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b5b90 con 0x7ff2c8158780 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cfed4700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c80b5e70 con 0x7ff2c80707d0 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cdc70700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80707d0 0x7ff2c80c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cd46f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2c80ccb30 0x7ff2c81e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cd46f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2c80ccb30 0x7ff2c81e2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35260/0 (socket says 172.21.15.79:35260) 2023-06-08T03:50:46.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2cd46f700 1 -- 172.21.15.79:0/827051231 learned_addr learned my addr 172.21.15.79:0/827051231 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7ff2ce471700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c8153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787affd700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f7868003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787a7fc700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7868004850 0x7f7868004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787b7fe700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7868005320 0x7f786800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787affd700 1 -- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7868004850 msgr2=0x7f7868004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787affd700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7868004850 0x7f7868004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787affd700 1 -- 172.21.15.79:0/747071189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7868005320 msgr2=0x7f786800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787affd700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7868005320 0x7f786800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787b7fe700 1 -- 172.21.15.79:0/747071189 reap_dead start 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f787affd700 1 -- 172.21.15.79:0/747071189 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c070ec0 con 0x7f7868003a00 2023-06-08T03:50:46.844 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f787affd700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f7868003dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f786400c2f0 tx=0x7f7868005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8e92fd4c6f89f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8418aef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f84140ae150 0x7f841406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8418aef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f841406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8418aef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841415a460 con 0x7f84140cc900 2023-06-08T03:50:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8418aef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8414069d50 con 0x7f84140ae150 2023-06-08T03:50:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8418aef700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8414069f60 con 0x7f84140cb1e0 2023-06-08T03:50:46.845 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f841259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f84140cc900 0x7f84140cccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60902/0 (socket says 172.21.15.79:60902) 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f841259c700 1 -- 172.21.15.79:0/1664773981 learned_addr learned my addr 172.21.15.79:0/1664773981 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8412d9d700 1 --2- 172.21.15.79:0/1664773981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f841406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.816+0000 7f8411d9b700 1 --2- 172.21.15.79:0/1664773981 >> v2:172.21.15.156:3300/0 conn(0x7f84140ae150 0x7f841406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f8412d9d700 1 -- 172.21.15.79:0/1664773981 >> v2:172.21.15.156:3300/0 conn(0x7f84140ae150 msgr2=0x7f841406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f8412d9d700 1 --2- 172.21.15.79:0/1664773981 >> v2:172.21.15.156:3300/0 conn(0x7f84140ae150 0x7f841406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f8412d9d700 1 -- 172.21.15.79:0/1664773981 >> v1:172.21.15.156:6789/0 conn(0x7f84140cc900 legacy=0x7f84140cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:46.846 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f8412d9d700 1 -- 172.21.15.79:0/1664773981 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841406a240 con 0x7f84140cb1e0 2023-06-08T03:50:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2ce471700 1 -- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80707d0 msgr2=0x7ff2c80c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2ce471700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80707d0 0x7ff2c80c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2ce471700 1 -- 172.21.15.79:0/827051231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2c80ccb30 msgr2=0x7ff2c81e2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2ce471700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2c80ccb30 0x7ff2c81e2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2ce471700 1 -- 172.21.15.79:0/827051231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80094a0 con 0x7ff2c8158780 2023-06-08T03:50:46.847 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400ce30 con 0x7f7868003a00 2023-06-08T03:50:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780054e0 con 0x7f3e900b4000 2023-06-08T03:50:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7800ce90 con 0x7f3e900b4000 2023-06-08T03:50:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780038c0 con 0x7f3e900b4000 2023-06-08T03:50:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e900b4fc0 con 0x7f3e900b4000 2023-06-08T03:50:46.848 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e901fa290 con 0x7f3e900b4000 2023-06-08T03:50:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2ce471700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c8153c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c780 tx=0x7ff2c000ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0003e10 con 0x7ff2c8158780 2023-06-08T03:50:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c00050d0 con 0x7ff2c8158780 2023-06-08T03:50:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c80b6150 con 0x7ff2c8158780 2023-06-08T03:50:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c81e2a60 con 0x7ff2c8158780 2023-06-08T03:50:46.849 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0005480 con 0x7ff2c8158780 2023-06-08T03:50:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864005960 con 0x7f7868003a00 2023-06-08T03:50:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f7879ffb700 1 -- 172.21.15.79:0/747071189 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864005d10 con 0x7f7868003a00 2023-06-08T03:50:46.850 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.817+0000 7f8412d9d700 1 --2- 172.21.15.79:0/1664773981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f841406aa40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f840c009d90 tx=0x7f840c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8835784a3c04faa6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe440012590 con 0x7fe4480ccfe0 2023-06-08T03:50:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 -- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 msgr2=0x7f7868003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.851 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f7868003dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f786400c2f0 tx=0x7f7868005880 comp rx=0 tx=0).stop 2023-06-08T03:50:46.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 -- 172.21.15.79:0/747071189 shutdown_connections 2023-06-08T03:50:46.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 --2- 172.21.15.79:0/747071189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f7868003dd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 -- 172.21.15.79:0/747071189 >> 172.21.15.79:0/747071189 conn(0x7f787c0ced30 msgr2=0x7f787c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 -- 172.21.15.79:0/747071189 shutdown_connections 2023-06-08T03:50:46.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f7881636700 1 -- 172.21.15.79:0/747071189 wait complete. 2023-06-08T03:50:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7fe43dffb700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe42c06acb0 0x7fe42c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe44008ab00 con 0x7fe4480ccfe0 2023-06-08T03:50:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90159b30 con 0x7f3e900b4000 2023-06-08T03:50:46.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b00052f0 con 0x7ff2c8158780 2023-06-08T03:50:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.819+0000 7f841159a700 1 -- 172.21.15.79:0/1664773981 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00e070 con 0x7f84140cb1e0 2023-06-08T03:50:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.819+0000 7f841159a700 1 -- 172.21.15.79:0/1664773981 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00bae0 con 0x7f84140cb1e0 2023-06-08T03:50:46.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.819+0000 7f841159a700 1 -- 172.21.15.79:0/1664773981 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00be70 con 0x7f84140cb1e0 2023-06-08T03:50:46.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.819+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f173c054b20 con 0x7f17440ad830 2023-06-08T03:50:46.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.818+0000 7fe44d51f700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe42c06acb0 0x7fe42c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.819+0000 7fe44d51f700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe42c06acb0 0x7fe42c06d160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe438009770 tx=0x7fe438006cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.855 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e78003a20 con 0x7f3e900b4000 2023-06-08T03:50:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff2c00055e0 con 0x7ff2c8158780 2023-06-08T03:50:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f8418aef700 1 -- 172.21.15.79:0/1664773981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 msgr2=0x7f841406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f8418aef700 1 --2- 172.21.15.79:0/1664773981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f841406aa40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f840c009d90 tx=0x7f840c00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:46.856 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f7881636700 1 Processor -- start 2023-06-08T03:50:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7ff2be7fc700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff2ac06ace0 0x7ff2ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff2c0088e00 con 0x7ff2c8158780 2023-06-08T03:50:46.857 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f3e8d7fa700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3e7006ace0 0x7f3e7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e78088780 con 0x7f3e900b4000 2023-06-08T03:50:46.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7ff2cdc70700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff2ac06ace0 0x7ff2ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f3e948e8700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3e7006ace0 0x7f3e7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7ff2cdc70700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff2ac06ace0 0x7ff2ac06d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b8010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.820+0000 7f7881636700 1 -- start start 2023-06-08T03:50:46.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f7881636700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f787c0c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f7881636700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 0x7f787c152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f7881636700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 0x7f787c1f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f7881636700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0b1e10 con 0x7f787c160d70 2023-06-08T03:50:46.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f7881636700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0c7a40 con 0x7f7868003a00 2023-06-08T03:50:46.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f7881636700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0c7760 con 0x7f787c1fa1a0 2023-06-08T03:50:46.860 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f3e948e8700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3e7006ace0 0x7f3e7006d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000f80 tx=0x7f3e88010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f787b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 0x7f787c1f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f787a7fc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 0x7f787c152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f787b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 0x7f787c1f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35280/0 (socket says 172.21.15.79:35280) 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f787b7fe700 1 -- 172.21.15.79:0/4199964597 learned_addr learned my addr 172.21.15.79:0/4199964597 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.821+0000 7f787a7fc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 0x7f787c152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49604/0 (socket says 172.21.15.79:49604) 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787a7fc700 1 -- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 msgr2=0x7f787c0c5e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787a7fc700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f787c0c5e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787a7fc700 1 -- 172.21.15.79:0/4199964597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 msgr2=0x7f787c1f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787a7fc700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 0x7f787c1f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787a7fc700 1 -- 172.21.15.79:0/4199964597 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0c8630 con 0x7f787c1fa1a0 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787b7fe700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 0x7f787c1f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f787a7fc700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 0x7f787c152b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f786c00f970 tx=0x7f786c00fc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00da60 con 0x7f787c1fa1a0 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00c040 con 0x7f787c1fa1a0 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00ddc0 con 0x7f787c1fa1a0 2023-06-08T03:50:46.863 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c0c8910 con 0x7f787c1fa1a0 2023-06-08T03:50:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.822+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c153060 con 0x7f787c1fa1a0 2023-06-08T03:50:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.823+0000 7f8418aef700 1 -- 172.21.15.79:0/1664773981 shutdown_connections 2023-06-08T03:50:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.823+0000 7f8418aef700 1 --2- 172.21.15.79:0/1664773981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f841406aa40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.823+0000 7f8418aef700 1 --2- 172.21.15.79:0/1664773981 >> v2:172.21.15.156:3300/0 conn(0x7f84140ae150 0x7f841406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.864 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.823+0000 7f8418aef700 1 -- 172.21.15.79:0/1664773981 >> 172.21.15.79:0/1664773981 conn(0x7f84140ced30 msgr2=0x7f84140cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.823+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c0c8490 con 0x7f787c1fa1a0 2023-06-08T03:50:46.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.824+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f786c013440 con 0x7f787c1fa1a0 2023-06-08T03:50:46.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.824+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe440055950 con 0x7fe4480ccfe0 2023-06-08T03:50:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.824+0000 7f78737fe700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f785c06ace0 0x7f785c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.824+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f786c0899e0 con 0x7f787c1fa1a0 2023-06-08T03:50:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.826+0000 7f8418aef700 1 -- 172.21.15.79:0/1664773981 shutdown_connections 2023-06-08T03:50:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.826+0000 7f8418aef700 1 -- 172.21.15.79:0/1664773981 wait complete. 2023-06-08T03:50:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.827+0000 7f8418aef700 1 Processor -- start 2023-06-08T03:50:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.827+0000 7f8418aef700 1 -- start start 2023-06-08T03:50:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8418aef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ae150 0x7f84141f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.867 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8418aef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f84140c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8418aef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140c5080 0x7f8414152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8418aef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84140c3c40 con 0x7f84140c5080 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8418aef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84140c3910 con 0x7f84140ae150 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8418aef700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84141f63e0 con 0x7f84140cb1e0 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8412d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140c5080 0x7f8414152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8412d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140c5080 0x7f8414152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35288/0 (socket says 172.21.15.79:35288) 2023-06-08T03:50:46.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8411d9b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f84140c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8412d9d700 1 -- 172.21.15.79:0/1635912657 learned_addr learned my addr 172.21.15.79:0/1635912657 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f787affd700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f785c06ace0 0x7f785c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8411d9b700 1 -- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ae150 msgr2=0x7f84141f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8411d9b700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ae150 0x7f84141f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8411d9b700 1 -- 172.21.15.79:0/1635912657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140c5080 msgr2=0x7f8414152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8411d9b700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140c5080 0x7f8414152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.828+0000 7f8411d9b700 1 -- 172.21.15.79:0/1635912657 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c009a70 con 0x7f84140cb1e0 2023-06-08T03:50:46.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.829+0000 7f787affd700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f785c06ace0 0x7f785c06d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7864000c00 tx=0x7f7864010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.829+0000 7f8411d9b700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f84140c62a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f840400e900 tx=0x7f840400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.830+0000 7f8764ab7700 1 Processor -- start 2023-06-08T03:50:46.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.830+0000 7f8764ab7700 1 -- start start 2023-06-08T03:50:46.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f8764ab7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876015a0e0 0x7f8760069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f8764ab7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8760071290 con 0x7f876006a290 2023-06-08T03:50:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f8764ab7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8760071570 con 0x7f87600cacc0 2023-06-08T03:50:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f8764ab7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600ca4c0 con 0x7f876015a0e0 2023-06-08T03:50:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f875dd9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f876006a290 0x7f87600707d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58768/0 (socket says 172.21.15.79:58768) 2023-06-08T03:50:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f875ed9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f87600cacc0 0x7f876006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:60932/0 (socket says 172.21.15.79:60932) 2023-06-08T03:50:46.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f875dd9b700 1 -- 172.21.15.79:0/1362636603 learned_addr learned my addr 172.21.15.79:0/1362636603 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:46.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3e90159b30 con 0x7f3e900b4000 2023-06-08T03:50:46.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.832+0000 7f875e59c700 1 --2- 172.21.15.79:0/1362636603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876015a0e0 0x7f8760069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400ee50 con 0x7f84140cb1e0 2023-06-08T03:50:46.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400c850 con 0x7f84140cb1e0 2023-06-08T03:50:46.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400cbc0 con 0x7f84140cb1e0 2023-06-08T03:50:46.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f875d59a700 1 -- 172.21.15.79:0/1362636603 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543963012 0 0) 0x7f8760071290 con 0x7f876006a290 2023-06-08T03:50:46.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f875d59a700 1 -- 172.21.15.79:0/1362636603 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87480035b0 con 0x7f876006a290 2023-06-08T03:50:46.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f875e59c700 1 -- 172.21.15.79:0/1362636603 >> v1:172.21.15.156:6789/0 conn(0x7f87600cacc0 legacy=0x7f876006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.874 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f875e59c700 1 -- 172.21.15.79:0/1362636603 >> v1:172.21.15.79:6789/0 conn(0x7f876006a290 legacy=0x7f87600707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f875e59c700 1 -- 172.21.15.79:0/1362636603 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600ca7a0 con 0x7f876015a0e0 2023-06-08T03:50:46.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f787c0c8490 con 0x7f787c1fa1a0 2023-06-08T03:50:46.875 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.833+0000 7f875e59c700 1 --2- 172.21.15.79:0/1362636603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876015a0e0 0x7f8760069d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8750009870 tx=0x7f8750009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=545232556f85aa4e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.834+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84141f66c0 con 0x7f84140cb1e0 2023-06-08T03:50:46.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.834+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84141f60d0 con 0x7f84140cb1e0 2023-06-08T03:50:46.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.836+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f840400cd20 con 0x7f84140cb1e0 2023-06-08T03:50:46.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.836+0000 7f8402ffd700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f83f406acb0 0x7f83f406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.836+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f84040896c0 con 0x7f84140cb1e0 2023-06-08T03:50:46.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.837+0000 7f841259c700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f83f406acb0 0x7f83f406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.837+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8414065ba0 con 0x7f84140cb1e0 2023-06-08T03:50:46.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.837+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff2c0053b30 con 0x7ff2c8158780 2023-06-08T03:50:46.877 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.837+0000 7f841259c700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f83f406acb0 0x7f83f406d160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84141659a0 tx=0x7f8408007b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.839+0000 7f875d59a700 1 -- 172.21.15.79:0/1362636603 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750011830 con 0x7f876015a0e0 2023-06-08T03:50:46.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.839+0000 7f875d59a700 1 -- 172.21.15.79:0/1362636603 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750011990 con 0x7f876015a0e0 2023-06-08T03:50:46.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.839+0000 7f875d59a700 1 -- 172.21.15.79:0/1362636603 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750011d20 con 0x7f876015a0e0 2023-06-08T03:50:46.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.840+0000 7f8764ab7700 1 -- 172.21.15.79:0/1362636603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876015a0e0 msgr2=0x7f8760069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.840+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1362636603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876015a0e0 0x7f8760069d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8750009870 tx=0x7f8750009b80 comp rx=0 tx=0).stop 2023-06-08T03:50:46.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.840+0000 7f8764ab7700 1 -- 172.21.15.79:0/1362636603 shutdown_connections 2023-06-08T03:50:46.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.840+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1362636603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876015a0e0 0x7f8760069d50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.840+0000 7f8764ab7700 1 -- 172.21.15.79:0/1362636603 >> 172.21.15.79:0/1362636603 conn(0x7f87600ced30 msgr2=0x7f87600cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:46.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.842+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8404057f80 con 0x7f84140cb1e0 2023-06-08T03:50:46.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.850+0000 7f8764ab7700 1 -- 172.21.15.79:0/1362636603 shutdown_connections 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.850+0000 7f8764ab7700 1 -- 172.21.15.79:0/1362636603 wait complete. 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.852+0000 7f8764ab7700 1 Processor -- start 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.852+0000 7f8764ab7700 1 -- start start 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.853+0000 7f8764ab7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 0x7f87600c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.853+0000 7f8764ab7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87600cacc0 0x7f87600c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.853+0000 7f8764ab7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876015a0e0 0x7f87600c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.853+0000 7f8764ab7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876015c1c0 con 0x7f87600cacc0 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.853+0000 7f8764ab7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876015c4a0 con 0x7f876015a0e0 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.853+0000 7f8764ab7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876015c780 con 0x7f876006a290 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 0x7f87600c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 0x7f87600c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49622/0 (socket says 172.21.15.79:49622) 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 -- 172.21.15.79:0/1479520161 learned_addr learned my addr 172.21.15.79:0/1479520161 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 -- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876015a0e0 msgr2=0x7f87600c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876015a0e0 0x7f87600c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 -- 172.21.15.79:0/1479520161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87600cacc0 msgr2=0x7f87600c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87600cacc0 0x7f87600c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 -- 172.21.15.79:0/1479520161 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500094a0 con 0x7f876006a290 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.854+0000 7f875e59c700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 0x7f87600c2860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f8750004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.859+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87500036a0 con 0x7f876006a290 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.859+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750003820 con 0x7f876006a290 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.859+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750003bd0 con 0x7f876006a290 2023-06-08T03:50:46.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.861+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876015ca60 con 0x7f876006a290 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.861+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876015cf80 con 0x7f876006a290 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.863+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876015cc80 con 0x7f876006a290 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.864+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8750003d30 con 0x7f876006a290 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.865+0000 7f874effd700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f874006adb0 0x7f874006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.865+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f875008abb0 con 0x7f876006a290 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.865+0000 7f875dd9b700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f874006adb0 0x7f874006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:46.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.867+0000 7f875dd9b700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f874006adb0 0x7f874006d260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8754007fd0 tx=0x7f8754007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:46.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:46.869+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8750055a80 con 0x7f876006a290 2023-06-08T03:50:47.190 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.186+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17440b2fe0 con 0x7f17440ad830 2023-06-08T03:50:47.191 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.187+0000 7f1740ff9700 1 -- 172.21.15.79:0/2646947848 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17440b2fe0 con 0x7f17440ad830 2023-06-08T03:50:47.194 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:47.194 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f173006ad30 msgr2=0x7f173006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f173006ad30 0x7f173006d1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17440ce370 tx=0x7f1734008030 comp rx=0 tx=0).stop 2023-06-08T03:50:47.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 msgr2=0x7f17440b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440b2200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f173c00eed0 tx=0x7f173c00b390 comp rx=0 tx=0).stop 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 shutdown_connections 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1744159f70 0x7f17441fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f173006ad30 0x7f173006d1e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440ad830 0x7f17440b2200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 --2- 172.21.15.79:0/2646947848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17440af160 0x7f17441fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 >> 172.21.15.79:0/2646947848 conn(0x7f17440ce4e0 msgr2=0x7f17440ab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 shutdown_connections 2023-06-08T03:50:47.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.191+0000 7f174a7cd700 1 -- 172.21.15.79:0/2646947848 wait complete. 2023-06-08T03:50:47.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.204+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa718005870 con 0x7fa71c003a00 2023-06-08T03:50:47.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.205+0000 7fa7267fc700 1 -- 172.21.15.79:0/470137892 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa72002a030 con 0x7fa71c003a00 2023-06-08T03:50:47.211 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:47.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.208+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa71406ace0 msgr2=0x7fa71406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.208+0000 7fa737f33700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa71406ace0 0x7fa71406d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa7300b0010 tx=0x7fa728009040 comp rx=0 tx=0).stop 2023-06-08T03:50:47.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.208+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 msgr2=0x7fa7301fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.208+0000 7fa737f33700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa7301fe100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa71c02e230 tx=0x7fa7200047c0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-06-08T03:50:47.214 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 shutdown_connections 2023-06-08T03:50:47.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa73015faf0 0x7fa73015ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa71c003a00 0x7fa7301fe100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa71406ace0 0x7fa71406d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 --2- 172.21.15.79:0/470137892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7300ae500 0x7fa7300ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 >> 172.21.15.79:0/470137892 conn(0x7fa730069d50 msgr2=0x7fa73006c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 shutdown_connections 2023-06-08T03:50:47.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.209+0000 7fa737f33700 1 -- 172.21.15.79:0/470137892 wait complete. 2023-06-08T03:50:47.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.216+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe4480b0750 con 0x7fe4480ccfe0 2023-06-08T03:50:47.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.217+0000 7fe43dffb700 1 -- 172.21.15.79:0/2261392571 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe440058f70 con 0x7fe4480ccfe0 2023-06-08T03:50:47.221 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:47.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.222+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f841406f1d0 con 0x7f84140cb1e0 2023-06-08T03:50:47.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.223+0000 7f8402ffd700 1 -- 172.21.15.79:0/1635912657 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8404021020 con 0x7f84140cb1e0 2023-06-08T03:50:47.227 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:47.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.223+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe42c06acb0 msgr2=0x7fe42c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.223+0000 7fe44f783700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe42c06acb0 0x7fe42c06d160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe438009770 tx=0x7fe438006cc0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.223+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 msgr2=0x7fe4480c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.223+0000 7fe44f783700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480c2d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe440015040 tx=0x7fe440004ad0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.224+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 shutdown_connections 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.224+0000 7fe44f783700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4480ccfe0 0x7fe4480c2d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.224+0000 7fe44f783700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4480ae300 0x7fe4481621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.224+0000 7fe44f783700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe42c06acb0 0x7fe42c06d160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.224+0000 7fe44f783700 1 --2- 172.21.15.79:0/2261392571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4480c14b0 0x7fe4480b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.224+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 >> 172.21.15.79:0/2261392571 conn(0x7fe4480ce4e0 msgr2=0x7fe4480aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.225+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f83f406acb0 msgr2=0x7f83f406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.225+0000 7f8418aef700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f83f406acb0 0x7f83f406d160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84141659a0 tx=0x7f8408007b30 comp rx=0 tx=0).stop 2023-06-08T03:50:47.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.225+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 msgr2=0x7f84140c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.225+0000 7f8418aef700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f84140c62a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f840400e900 tx=0x7f840400ec10 comp rx=0 tx=0).stop 2023-06-08T03:50:47.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.227+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 shutdown_connections 2023-06-08T03:50:47.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.227+0000 7fe44f783700 1 -- 172.21.15.79:0/2261392571 wait complete. 2023-06-08T03:50:47.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-06-08T03:50:47.231 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 shutdown_connections 2023-06-08T03:50:47.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84140c5080 0x7f8414152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84140ae150 0x7f84141f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f83f406acb0 0x7f83f406d160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 --2- 172.21.15.79:0/1635912657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84140cb1e0 0x7f84140c62a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 >> 172.21.15.79:0/1635912657 conn(0x7f84140ced30 msgr2=0x7f84140c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff2b00055a0 con 0x7ff2c8158780 2023-06-08T03:50:47.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 shutdown_connections 2023-06-08T03:50:47.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.231+0000 7f8418aef700 1 -- 172.21.15.79:0/1635912657 wait complete. 2023-06-08T03:50:47.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.232+0000 7ff2be7fc700 1 -- 172.21.15.79:0/827051231 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff2c0017020 con 0x7ff2c8158780 2023-06-08T03:50:47.237 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:47.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff2ac06ace0 msgr2=0x7ff2ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff2ac06ace0 0x7ff2ac06d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8000c00 tx=0x7ff2b8010000 comp rx=0 tx=0).stop 2023-06-08T03:50:47.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 msgr2=0x7ff2c8153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c8153c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000c780 tx=0x7ff2c000ca90 comp rx=0 tx=0).stop 2023-06-08T03:50:47.238 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 shutdown_connections 2023-06-08T03:50:47.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2c8158780 0x7ff2c8153c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff2ac06ace0 0x7ff2ac06d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2c80707d0 0x7ff2c80c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 --2- 172.21.15.79:0/827051231 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2c80ccb30 0x7ff2c81e2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.239 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.234+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 >> 172.21.15.79:0/827051231 conn(0x7ff2c80ced30 msgr2=0x7ff2c80afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.239+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 shutdown_connections 2023-06-08T03:50:47.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.238+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f787c0c8490 con 0x7f787c1fa1a0 2023-06-08T03:50:47.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.239+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3e90159b30 con 0x7f3e900b4000 2023-06-08T03:50:47.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.239+0000 7ff2cfed4700 1 -- 172.21.15.79:0/827051231 wait complete. 2023-06-08T03:50:47.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.239+0000 7f3e8d7fa700 1 -- 172.21.15.79:0/2983784726 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e90159b30 con 0x7f3e900b4000 2023-06-08T03:50:47.244 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2023-06-08T03:50:47.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.240+0000 7f78737fe700 1 -- 172.21.15.79:0/4199964597 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f787c0c8490 con 0x7f787c1fa1a0 2023-06-08T03:50:47.245 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2023-06-08T03:50:47.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-06-08T03:50:47.246 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.242+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3e7006ace0 msgr2=0x7f3e7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.242+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3e7006ace0 0x7f3e7006d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e88000f80 tx=0x7f3e88010000 comp rx=0 tx=0).stop 2023-06-08T03:50:47.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.242+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 msgr2=0x7f3e900c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.242+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 0x7f3e900c6c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c4c0 tx=0x7f3e7800c7d0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 shutdown_connections 2023-06-08T03:50:47.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e901f98d0 0x7f3e901f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3e7006ace0 0x7f3e7006d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e7c003a00 0x7f3e90155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 --2- 172.21.15.79:0/2983784726 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e900b4000 0x7f3e900c6c00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 >> 172.21.15.79:0/2983784726 conn(0x7f3e900b1d30 msgr2=0x7f3e900cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 shutdown_connections 2023-06-08T03:50:47.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.243+0000 7f3e96b4c700 1 -- 172.21.15.79:0/2983784726 wait complete. 2023-06-08T03:50:47.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.244+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f785c06ace0 msgr2=0x7f785c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.244+0000 7f7881636700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f785c06ace0 0x7f785c06d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7864000c00 tx=0x7f7864010000 comp rx=0 tx=0).stop 2023-06-08T03:50:47.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.244+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 msgr2=0x7f787c152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.244+0000 7f7881636700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 0x7f787c152b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f786c00f970 tx=0x7f786c00fc80 comp rx=0 tx=0).stop 2023-06-08T03:50:47.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 shutdown_connections 2023-06-08T03:50:47.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f787c160d70 0x7f787c1f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7868003a00 0x7f787c0c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f785c06ace0 0x7f785c06d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 --2- 172.21.15.79:0/4199964597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f787c1fa1a0 0x7f787c152b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 >> 172.21.15.79:0/4199964597 conn(0x7f787c0ced30 msgr2=0x7f787c15fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 shutdown_connections 2023-06-08T03:50:47.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.247+0000 7f7881636700 1 -- 172.21.15.79:0/4199964597 wait complete. 2023-06-08T03:50:47.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-06-08T03:50:47.256 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-06-08T03:50:47.262 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-06-08T03:50:47.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.258+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f876006f1d0 con 0x7f876006a290 2023-06-08T03:50:47.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.259+0000 7f874effd700 1 -- 172.21.15.79:0/1479520161 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87500590a0 con 0x7f876006a290 2023-06-08T03:50:47.264 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.264 INFO:teuthology.orchestra.run.smithi079.stdout:55834574855 2023-06-08T03:50:47.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f874006adb0 msgr2=0x7f874006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f874006adb0 0x7f874006d260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8754007fd0 tx=0x7f8754007980 comp rx=0 tx=0).stop 2023-06-08T03:50:47.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 msgr2=0x7f87600c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 0x7f87600c2860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8750000c00 tx=0x7f8750004750 comp rx=0 tx=0).stop 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 shutdown_connections 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876015a0e0 0x7f87600c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f874006adb0 0x7f874006d260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876006a290 0x7f87600c2860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 --2- 172.21.15.79:0/1479520161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87600cacc0 0x7f87600c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 >> 172.21.15.79:0/1479520161 conn(0x7f87600ced30 msgr2=0x7f87600ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 shutdown_connections 2023-06-08T03:50:47.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.261+0000 7f8764ab7700 1 -- 172.21.15.79:0/1479520161 wait complete. 2023-06-08T03:50:47.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-06-08T03:50:47.268 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-06-08T03:50:47.277 DEBUG:teuthology.parallel:result is None 2023-06-08T03:50:47.278 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-08T03:50:47.278 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-08T03:50:47.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 Processor -- start 2023-06-08T03:50:47.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 -- start start 2023-06-08T03:50:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f405c06a460 0x7f405c071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 0x7f405c071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405c0cee20 con 0x7f405c15f6c0 2023-06-08T03:50:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405c0cf330 con 0x7f405c06a460 2023-06-08T03:50:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.434+0000 7f4062fc6700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c06fa00 con 0x7f405c0cd6c0 2023-06-08T03:50:47.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f4060d62700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f405c15f6c0 0x7f405c069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58784/0 (socket says 172.21.15.79:58784) 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f4060d62700 1 -- 172.21.15.79:0/3396069583 learned_addr learned my addr 172.21.15.79:0/3396069583 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405bfff700 1 --2- 172.21.15.79:0/3396069583 >> v2:172.21.15.156:3300/0 conn(0x7f405c06a460 0x7f405c071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f4061563700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 0x7f405c071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339910392 0 0) 0x7f405c0cee20 con 0x7f405c15f6c0 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f404c0035b0 con 0x7f405c15f6c0 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 545492225 0 0) 0x7f404c0035b0 con 0x7f405c15f6c0 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 msgr2=0x7f405c071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:47.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 0x7f405c071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 >> v2:172.21.15.156:3300/0 conn(0x7f405c06a460 msgr2=0x7f405c071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 --2- 172.21.15.79:0/3396069583 >> v2:172.21.15.156:3300/0 conn(0x7f405c06a460 0x7f405c071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c0cd1e0 con 0x7f405c15f6c0 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f4050001670 con 0x7f405c15f6c0 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 >> v1:172.21.15.79:6789/0 conn(0x7f405c15f6c0 legacy=0x7f405c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 0x7f404c003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c003ff0 0x7f404c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c004d90 0x7f404c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0cf330 con 0x7f404c004d90 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c06fa00 con 0x7f404c003ff0 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c00ba10 con 0x7f405c0cd6c0 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f4061563700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c004d90 0x7f404c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f4060d62700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 0x7f404c003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.435+0000 7f405bfff700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c003ff0 0x7f404c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 -- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 msgr2=0x7f404c003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0cd6c0 0x7f404c003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 -- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c003ff0 msgr2=0x7f404c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c003ff0 0x7f404c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 -- 172.21.15.79:0/3396069583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c0035b0 con 0x7f404c004d90 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c004d90 0x7f404c005200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f405400e170 tx=0x7f404c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29338961c833483e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405400ec90 con 0x7f404c004d90 2023-06-08T03:50:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405400f040 con 0x7f404c004d90 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f405b7fe700 1 -- 172.21.15.79:0/3396069583 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054014480 con 0x7f404c004d90 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4062fc6700 1 -- 172.21.15.79:0/3396069583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c004d90 msgr2=0x7f404c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4062fc6700 1 --2- 172.21.15.79:0/3396069583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f404c004d90 0x7f404c005200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f405400e170 tx=0x7f404c005880 comp rx=0 tx=0).stop 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4061563700 1 -- 172.21.15.79:0/3396069583 reap_dead start 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4062fc6700 1 -- 172.21.15.79:0/3396069583 shutdown_connections 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4062fc6700 1 -- 172.21.15.79:0/3396069583 >> 172.21.15.79:0/3396069583 conn(0x7f405c0b1d20 msgr2=0x7f405c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4062fc6700 1 -- 172.21.15.79:0/3396069583 shutdown_connections 2023-06-08T03:50:47.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.436+0000 7f4062fc6700 1 -- 172.21.15.79:0/3396069583 wait complete. 2023-06-08T03:50:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 Processor -- start 2023-06-08T03:50:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 -- start start 2023-06-08T03:50:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c004d90 0x7f405c0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0719a0 0x7f405c0c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 0x7f405c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c071840 con 0x7f405c0c53f0 2023-06-08T03:50:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0c7460 con 0x7f404c004d90 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.438+0000 7f4062fc6700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405c0c67e0 con 0x7f405c0719a0 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 0x7f405c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 0x7f405c15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35324/0 (socket says 172.21.15.79:35324) 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 -- 172.21.15.79:0/317416545 learned_addr learned my addr 172.21.15.79:0/317416545 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4060d62700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c004d90 0x7f405c0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f405bfff700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0719a0 0x7f405c0c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 -- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0719a0 msgr2=0x7f405c0c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0719a0 0x7f405c0c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 -- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c004d90 msgr2=0x7f405c0c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c004d90 0x7f405c0c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 -- 172.21.15.79:0/317416545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c0c6ac0 con 0x7f405c0c53f0 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4061563700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 0x7f405c15c7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4054016000 tx=0x7f4054009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054007000 con 0x7f405c0c53f0 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054004450 con 0x7f405c0c53f0 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c0c6da0 con 0x7f405c0c53f0 2023-06-08T03:50:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054004800 con 0x7f405c0c53f0 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.439+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c15cd00 con 0x7f405c0c53f0 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.440+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405c0c75a0 con 0x7f405c0c53f0 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.441+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4054023070 con 0x7f405c0c53f0 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.441+0000 7f40597fa700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f404006ace0 0x7f404006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.442+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f405408b0f0 con 0x7f405c0c53f0 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.442+0000 7f4060d62700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f404006ace0 0x7f404006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.442+0000 7f4060d62700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f404006ace0 0x7f404006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4050000fb0 tx=0x7f4050009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.445+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4054056020 con 0x7f405c0c53f0 2023-06-08T03:50:47.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.628+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 --> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f405c160a30 con 0x7f404006ace0 2023-06-08T03:50:47.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.639+0000 7f40597fa700 1 -- 172.21.15.79:0/317416545 <== mgr.4099 v2:172.21.15.156:6832/2537078349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367585 (secure 0 0 0) 0x7f4050007030 con 0x7f404006ace0 2023-06-08T03:50:47.646 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f404006ace0 msgr2=0x7f404006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f404006ace0 0x7f404006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4050000fb0 tx=0x7f4050009040 comp rx=0 tx=0).stop 2023-06-08T03:50:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 msgr2=0x7f405c15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 0x7f405c15c7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4054016000 tx=0x7f4054009750 comp rx=0 tx=0).stop 2023-06-08T03:50:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 shutdown_connections 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f405c0c53f0 0x7f405c15c7c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f404c004d90 0x7f405c0c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f404006ace0 0x7f404006d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 --2- 172.21.15.79:0/317416545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405c0719a0 0x7f405c0c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 >> 172.21.15.79:0/317416545 conn(0x7f405c0b1d20 msgr2=0x7f405c06eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 shutdown_connections 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.642+0000 7f4062fc6700 1 -- 172.21.15.79:0/317416545 wait complete. 2023-06-08T03:50:47.649 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-08T03:50:47.658 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-06-08T03:50:45.874343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218920,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755800,"statfs":{"total":773094113280,"available":772869939200,"internally_reserved":0,"allocated":3383296,"data_stored":2065192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003688"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312366+0000","last_change":"2023-06-08T03:50:24.695168+0000","last_active":"2023-06-08T03:50:29.312366+0000","last_peered":"2023-06-08T03:50:29.312366+0000","last_clean":"2023-06-08T03:50:29.312366+0000","last_became_active":"2023-06-08T03:50:24.694872+0000","last_became_peered":"2023-06-08T03:50:24.694872+0000","last_unstale":"2023-06-08T03:50:29.312366+0000","last_undegraded":"2023-06-08T03:50:29.312366+0000","last_fullsized":"2023-06-08T03:50:29.312366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791005+0000","last_change":"2023-06-08T03:50:24.687043+0000","last_active":"2023-06-08T03:50:29.791005+0000","last_peered":"2023-06-08T03:50:29.791005+0000","last_clean":"2023-06-08T03:50:29.791005+0000","last_became_active":"2023-06-08T03:50:24.686710+0000","last_became_peered":"2023-06-08T03:50:24.686710+0000","last_unstale":"2023-06-08T03:50:29.791005+0000","last_undegraded":"2023-06-08T03:50:29.791005+0000","last_fullsized":"2023-06-08T03:50:29.791005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591260+0000","last_change":"2023-06-08T03:50:24.686759+0000","last_active":"2023-06-08T03:50:29.591260+0000","last_peered":"2023-06-08T03:50:29.591260+0000","last_clean":"2023-06-08T03:50:29.591260+0000","last_became_active":"2023-06-08T03:50:24.686497+0000","last_became_peered":"2023-06-08T03:50:24.686497+0000","last_unstale":"2023-06-08T03:50:29.591260+0000","last_undegraded":"2023-06-08T03:50:29.591260+0000","last_fullsized":"2023-06-08T03:50:29.591260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:14.651224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591320+0000","last_change":"2023-06-08T03:50:24.691468+0000","last_active":"2023-06-08T03:50:29.591320+0000","last_peered":"2023-06-08T03:50:29.591320+0000","last_clean":"2023-06-08T03:50:29.591320+0000","last_became_active":"2023-06-08T03:50:24.691218+0000","last_became_peered":"2023-06-08T03:50:24.691218+0000","last_unstale":"2023-06-08T03:50:29.591320+0000","last_undegraded":"2023-06-08T03:50:29.591320+0000","last_fullsized":"2023-06-08T03:50:29.591320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309624+0000","last_change":"2023-06-08T03:50:24.685604+0000","last_active":"2023-06-08T03:50:29.309624+0000","last_peered":"2023-06-08T03:50:29.309624+0000","last_clean":"2023-06-08T03:50:29.309624+0000","last_became_active":"2023-06-08T03:50:24.685184+0000","last_became_peered":"2023-06-08T03:50:24.685184+0000","last_unstale":"2023-06-08T03:50:29.309624+0000","last_undegraded":"2023-06-08T03:50:29.309624+0000","last_fullsized":"2023-06-08T03:50:29.309624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:09:46.870511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789772+0000","last_change":"2023-06-08T03:50:24.698386+0000","last_active":"2023-06-08T03:50:29.789772+0000","last_peered":"2023-06-08T03:50:29.789772+0000","last_clean":"2023-06-08T03:50:29.789772+0000","last_became_active":"2023-06-08T03:50:24.698086+0000","last_became_peered":"2023-06-08T03:50:24.698086+0000","last_unstale":"2023-06-08T03:50:29.789772+0000","last_undegraded":"2023-06-08T03:50:29.789772+0000","last_fullsized":"2023-06-08T03:50:29.789772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:11.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790111+0000","last_change":"2023-06-08T03:50:24.693046+0000","last_active":"2023-06-08T03:50:29.790111+0000","last_peered":"2023-06-08T03:50:29.790111+0000","last_clean":"2023-06-08T03:50:29.790111+0000","last_became_active":"2023-06-08T03:50:24.692581+0000","last_became_peered":"2023-06-08T03:50:24.692581+0000","last_unstale":"2023-06-08T03:50:29.790111+0000","last_undegraded":"2023-06-08T03:50:29.790111+0000","last_fullsized":"2023-06-08T03:50:29.790111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:39.113365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790425+0000","last_change":"2023-06-08T03:50:24.689352+0000","last_active":"2023-06-08T03:50:29.790425+0000","last_peered":"2023-06-08T03:50:29.790425+0000","last_clean":"2023-06-08T03:50:29.790425+0000","last_became_active":"2023-06-08T03:50:24.688961+0000","last_became_peered":"2023-06-08T03:50:24.688961+0000","last_unstale":"2023-06-08T03:50:29.790425+0000","last_undegraded":"2023-06-08T03:50:29.790425+0000","last_fullsized":"2023-06-08T03:50:29.790425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:59:48.614865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310084+0000","last_change":"2023-06-08T03:50:24.682755+0000","last_active":"2023-06-08T03:50:29.310084+0000","last_peered":"2023-06-08T03:50:29.310084+0000","last_clean":"2023-06-08T03:50:29.310084+0000","last_became_active":"2023-06-08T03:50:24.682417+0000","last_became_peered":"2023-06-08T03:50:24.682417+0000","last_unstale":"2023-06-08T03:50:29.310084+0000","last_undegraded":"2023-06-08T03:50:29.310084+0000","last_fullsized":"2023-06-08T03:50:29.310084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:38:17.580156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791038+0000","last_change":"2023-06-08T03:50:24.697828+0000","last_active":"2023-06-08T03:50:29.791038+0000","last_peered":"2023-06-08T03:50:29.791038+0000","last_clean":"2023-06-08T03:50:29.791038+0000","last_became_active":"2023-06-08T03:50:24.697576+0000","last_became_peered":"2023-06-08T03:50:24.697576+0000","last_unstale":"2023-06-08T03:50:29.791038+0000","last_undegraded":"2023-06-08T03:50:29.791038+0000","last_fullsized":"2023-06-08T03:50:29.791038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789439+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.789439+0000","last_peered":"2023-06-08T03:50:29.789439+0000","last_clean":"2023-06-08T03:50:29.789439+0000","last_became_active":"2023-06-08T03:50:24.688197+0000","last_became_peered":"2023-06-08T03:50:24.688197+0000","last_unstale":"2023-06-08T03:50:29.789439+0000","last_undegraded":"2023-06-08T03:50:29.789439+0000","last_fullsized":"2023-06-08T03:50:29.789439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:04:27.557616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311848+0000","last_change":"2023-06-08T03:50:24.692867+0000","last_active":"2023-06-08T03:50:29.311848+0000","last_peered":"2023-06-08T03:50:29.311848+0000","last_clean":"2023-06-08T03:50:29.311848+0000","last_became_active":"2023-06-08T03:50:24.692613+0000","last_became_peered":"2023-06-08T03:50:24.692613+0000","last_unstale":"2023-06-08T03:50:29.311848+0000","last_undegraded":"2023-06-08T03:50:29.311848+0000","last_fullsized":"2023-06-08T03:50:29.311848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:02.122906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309156+0000","last_change":"2023-06-08T03:50:24.694092+0000","last_active":"2023-06-08T03:50:29.309156+0000","last_peered":"2023-06-08T03:50:29.309156+0000","last_clean":"2023-06-08T03:50:29.309156+0000","last_became_active":"2023-06-08T03:50:24.693783+0000","last_became_peered":"2023-06-08T03:50:24.693783+0000","last_unstale":"2023-06-08T03:50:29.309156+0000","last_undegraded":"2023-06-08T03:50:29.309156+0000","last_fullsized":"2023-06-08T03:50:29.309156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790036+0000","last_change":"2023-06-08T03:50:24.702721+0000","last_active":"2023-06-08T03:50:29.790036+0000","last_peered":"2023-06-08T03:50:29.790036+0000","last_clean":"2023-06-08T03:50:29.790036+0000","last_became_active":"2023-06-08T03:50:24.702438+0000","last_became_peered":"2023-06-08T03:50:24.702438+0000","last_unstale":"2023-06-08T03:50:29.790036+0000","last_undegraded":"2023-06-08T03:50:29.790036+0000","last_fullsized":"2023-06-08T03:50:29.790036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:59:10.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.692560+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.692035+0000","last_became_peered":"2023-06-08T03:50:24.692035+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790991+0000","last_change":"2023-06-08T03:50:24.698511+0000","last_active":"2023-06-08T03:50:29.790991+0000","last_peered":"2023-06-08T03:50:29.790991+0000","last_clean":"2023-06-08T03:50:29.790991+0000","last_became_active":"2023-06-08T03:50:24.698281+0000","last_became_peered":"2023-06-08T03:50:24.698281+0000","last_unstale":"2023-06-08T03:50:29.790991+0000","last_undegraded":"2023-06-08T03:50:29.790991+0000","last_fullsized":"2023-06-08T03:50:29.790991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:27:46.498562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789941+0000","last_change":"2023-06-08T03:50:24.695969+0000","last_active":"2023-06-08T03:50:29.789941+0000","last_peered":"2023-06-08T03:50:29.789941+0000","last_clean":"2023-06-08T03:50:29.789941+0000","last_became_active":"2023-06-08T03:50:24.695711+0000","last_became_peered":"2023-06-08T03:50:24.695711+0000","last_unstale":"2023-06-08T03:50:29.789941+0000","last_undegraded":"2023-06-08T03:50:29.789941+0000","last_fullsized":"2023-06-08T03:50:29.789941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:45:45.278948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591321+0000","last_change":"2023-06-08T03:50:24.690923+0000","last_active":"2023-06-08T03:50:29.591321+0000","last_peered":"2023-06-08T03:50:29.591321+0000","last_clean":"2023-06-08T03:50:29.591321+0000","last_became_active":"2023-06-08T03:50:24.690667+0000","last_became_peered":"2023-06-08T03:50:24.690667+0000","last_unstale":"2023-06-08T03:50:29.591321+0000","last_undegraded":"2023-06-08T03:50:29.591321+0000","last_fullsized":"2023-06-08T03:50:29.591321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311961+0000","last_change":"2023-06-08T03:50:24.701274+0000","last_active":"2023-06-08T03:50:29.311961+0000","last_peered":"2023-06-08T03:50:29.311961+0000","last_clean":"2023-06-08T03:50:29.311961+0000","last_became_active":"2023-06-08T03:50:24.700991+0000","last_became_peered":"2023-06-08T03:50:24.700991+0000","last_unstale":"2023-06-08T03:50:29.311961+0000","last_undegraded":"2023-06-08T03:50:29.311961+0000","last_fullsized":"2023-06-08T03:50:29.311961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789321+0000","last_change":"2023-06-08T03:50:24.690034+0000","last_active":"2023-06-08T03:50:29.789321+0000","last_peered":"2023-06-08T03:50:29.789321+0000","last_clean":"2023-06-08T03:50:29.789321+0000","last_became_active":"2023-06-08T03:50:24.689524+0000","last_became_peered":"2023-06-08T03:50:24.689524+0000","last_unstale":"2023-06-08T03:50:29.789321+0000","last_undegraded":"2023-06-08T03:50:29.789321+0000","last_fullsized":"2023-06-08T03:50:29.789321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790401+0000","last_change":"2023-06-08T03:50:24.702598+0000","last_active":"2023-06-08T03:50:29.790401+0000","last_peered":"2023-06-08T03:50:29.790401+0000","last_clean":"2023-06-08T03:50:29.790401+0000","last_became_active":"2023-06-08T03:50:24.702337+0000","last_became_peered":"2023-06-08T03:50:24.702337+0000","last_unstale":"2023-06-08T03:50:29.790401+0000","last_undegraded":"2023-06-08T03:50:29.790401+0000","last_fullsized":"2023-06-08T03:50:29.790401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:29:18.553774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789291+0000","last_change":"2023-06-08T03:50:24.689664+0000","last_active":"2023-06-08T03:50:29.789291+0000","last_peered":"2023-06-08T03:50:29.789291+0000","last_clean":"2023-06-08T03:50:29.789291+0000","last_became_active":"2023-06-08T03:50:24.689386+0000","last_became_peered":"2023-06-08T03:50:24.689386+0000","last_unstale":"2023-06-08T03:50:29.789291+0000","last_undegraded":"2023-06-08T03:50:29.789291+0000","last_fullsized":"2023-06-08T03:50:29.789291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:23:34.826399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308547+0000","last_change":"2023-06-08T03:50:24.690551+0000","last_active":"2023-06-08T03:50:29.308547+0000","last_peered":"2023-06-08T03:50:29.308547+0000","last_clean":"2023-06-08T03:50:29.308547+0000","last_became_active":"2023-06-08T03:50:24.689459+0000","last_became_peered":"2023-06-08T03:50:24.689459+0000","last_unstale":"2023-06-08T03:50:29.308547+0000","last_undegraded":"2023-06-08T03:50:29.308547+0000","last_fullsized":"2023-06-08T03:50:29.308547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:21:13.898688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789394+0000","last_change":"2023-06-08T03:50:24.702787+0000","last_active":"2023-06-08T03:50:29.789394+0000","last_peered":"2023-06-08T03:50:29.789394+0000","last_clean":"2023-06-08T03:50:29.789394+0000","last_became_active":"2023-06-08T03:50:24.702390+0000","last_became_peered":"2023-06-08T03:50:24.702390+0000","last_unstale":"2023-06-08T03:50:29.789394+0000","last_undegraded":"2023-06-08T03:50:29.789394+0000","last_fullsized":"2023-06-08T03:50:29.789394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310288+0000","last_change":"2023-06-08T03:50:24.689083+0000","last_active":"2023-06-08T03:50:29.310288+0000","last_peered":"2023-06-08T03:50:29.310288+0000","last_clean":"2023-06-08T03:50:29.310288+0000","last_became_active":"2023-06-08T03:50:24.688803+0000","last_became_peered":"2023-06-08T03:50:24.688803+0000","last_unstale":"2023-06-08T03:50:29.310288+0000","last_undegraded":"2023-06-08T03:50:29.310288+0000","last_fullsized":"2023-06-08T03:50:29.310288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:59:04.642632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789549+0000","last_change":"2023-06-08T03:50:24.689752+0000","last_active":"2023-06-08T03:50:29.789549+0000","last_peered":"2023-06-08T03:50:29.789549+0000","last_clean":"2023-06-08T03:50:29.789549+0000","last_became_active":"2023-06-08T03:50:24.689199+0000","last_became_peered":"2023-06-08T03:50:24.689199+0000","last_unstale":"2023-06-08T03:50:29.789549+0000","last_undegraded":"2023-06-08T03:50:29.789549+0000","last_fullsized":"2023-06-08T03:50:29.789549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:06:49.219275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790465+0000","last_change":"2023-06-08T03:50:24.699485+0000","last_active":"2023-06-08T03:50:29.790465+0000","last_peered":"2023-06-08T03:50:29.790465+0000","last_clean":"2023-06-08T03:50:29.790465+0000","last_became_active":"2023-06-08T03:50:24.699155+0000","last_became_peered":"2023-06-08T03:50:24.699155+0000","last_unstale":"2023-06-08T03:50:29.790465+0000","last_undegraded":"2023-06-08T03:50:29.790465+0000","last_fullsized":"2023-06-08T03:50:29.790465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:09:23.122980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790702+0000","last_change":"2023-06-08T03:50:28.917312+0000","last_active":"2023-06-08T03:50:29.790702+0000","last_peered":"2023-06-08T03:50:29.790702+0000","last_clean":"2023-06-08T03:50:29.790702+0000","last_became_active":"2023-06-08T03:50:28.916901+0000","last_became_peered":"2023-06-08T03:50:28.916901+0000","last_unstale":"2023-06-08T03:50:29.790702+0000","last_undegraded":"2023-06-08T03:50:29.790702+0000","last_fullsized":"2023-06-08T03:50:29.790702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6,0],"acting":[2,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789907+0000","last_change":"2023-06-08T03:50:24.285058+0000","last_active":"2023-06-08T03:50:29.789907+0000","last_peered":"2023-06-08T03:50:29.789907+0000","last_clean":"2023-06-08T03:50:29.789907+0000","last_became_active":"2023-06-08T03:50:24.284770+0000","last_became_peered":"2023-06-08T03:50:24.284770+0000","last_unstale":"2023-06-08T03:50:29.789907+0000","last_undegraded":"2023-06-08T03:50:29.789907+0000","last_fullsized":"2023-06-08T03:50:29.789907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:16:20.267305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789601+0000","last_change":"2023-06-08T03:50:24.698385+0000","last_active":"2023-06-08T03:50:29.789601+0000","last_peered":"2023-06-08T03:50:29.789601+0000","last_clean":"2023-06-08T03:50:29.789601+0000","last_became_active":"2023-06-08T03:50:24.697966+0000","last_became_peered":"2023-06-08T03:50:24.697966+0000","last_unstale":"2023-06-08T03:50:29.789601+0000","last_undegraded":"2023-06-08T03:50:29.789601+0000","last_fullsized":"2023-06-08T03:50:29.789601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:20.947165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791158+0000","last_change":"2023-06-08T03:50:28.922104+0000","last_active":"2023-06-08T03:50:29.791158+0000","last_peered":"2023-06-08T03:50:29.791158+0000","last_clean":"2023-06-08T03:50:29.791158+0000","last_became_active":"2023-06-08T03:50:28.921721+0000","last_became_peered":"2023-06-08T03:50:28.921721+0000","last_unstale":"2023-06-08T03:50:29.791158+0000","last_undegraded":"2023-06-08T03:50:29.791158+0000","last_fullsized":"2023-06-08T03:50:29.791158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:28.031781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,5],"acting":[2,7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308030+0000","last_change":"2023-06-08T03:50:24.286192+0000","last_active":"2023-06-08T03:50:29.308030+0000","last_peered":"2023-06-08T03:50:29.308030+0000","last_clean":"2023-06-08T03:50:29.308030+0000","last_became_active":"2023-06-08T03:50:24.285939+0000","last_became_peered":"2023-06-08T03:50:24.285939+0000","last_unstale":"2023-06-08T03:50:29.308030+0000","last_undegraded":"2023-06-08T03:50:29.308030+0000","last_fullsized":"2023-06-08T03:50:29.308030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:32.185279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308525+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.308525+0000","last_peered":"2023-06-08T03:50:29.308525+0000","last_clean":"2023-06-08T03:50:29.308525+0000","last_became_active":"2023-06-08T03:50:24.688036+0000","last_became_peered":"2023-06-08T03:50:24.688036+0000","last_unstale":"2023-06-08T03:50:29.308525+0000","last_undegraded":"2023-06-08T03:50:29.308525+0000","last_fullsized":"2023-06-08T03:50:29.308525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:23:40.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309722+0000","last_change":"2023-06-08T03:50:28.921813+0000","last_active":"2023-06-08T03:50:29.309722+0000","last_peered":"2023-06-08T03:50:29.309722+0000","last_clean":"2023-06-08T03:50:29.309722+0000","last_became_active":"2023-06-08T03:50:28.921392+0000","last_became_peered":"2023-06-08T03:50:28.921392+0000","last_unstale":"2023-06-08T03:50:29.309722+0000","last_undegraded":"2023-06-08T03:50:29.309722+0000","last_fullsized":"2023-06-08T03:50:29.309722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:18.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2,7],"acting":[6,3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310612+0000","last_change":"2023-06-08T03:50:24.281038+0000","last_active":"2023-06-08T03:50:29.310612+0000","last_peered":"2023-06-08T03:50:29.310612+0000","last_clean":"2023-06-08T03:50:29.310612+0000","last_became_active":"2023-06-08T03:50:24.280771+0000","last_became_peered":"2023-06-08T03:50:24.280771+0000","last_unstale":"2023-06-08T03:50:29.310612+0000","last_undegraded":"2023-06-08T03:50:29.310612+0000","last_fullsized":"2023-06-08T03:50:29.310612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:43.269177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790403+0000","last_change":"2023-06-08T03:50:24.686123+0000","last_active":"2023-06-08T03:50:29.790403+0000","last_peered":"2023-06-08T03:50:29.790403+0000","last_clean":"2023-06-08T03:50:29.790403+0000","last_became_active":"2023-06-08T03:50:24.685951+0000","last_became_peered":"2023-06-08T03:50:24.685951+0000","last_unstale":"2023-06-08T03:50:29.790403+0000","last_undegraded":"2023-06-08T03:50:29.790403+0000","last_fullsized":"2023-06-08T03:50:29.790403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:44:16.167507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789665+0000","last_change":"2023-06-08T03:50:28.929521+0000","last_active":"2023-06-08T03:50:29.789665+0000","last_peered":"2023-06-08T03:50:29.789665+0000","last_clean":"2023-06-08T03:50:29.789665+0000","last_became_active":"2023-06-08T03:50:28.929194+0000","last_became_peered":"2023-06-08T03:50:28.929194+0000","last_unstale":"2023-06-08T03:50:29.789665+0000","last_undegraded":"2023-06-08T03:50:29.789665+0000","last_fullsized":"2023-06-08T03:50:29.789665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:19:17.680617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,5],"acting":[3,2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312228+0000","last_change":"2023-06-08T03:50:24.278075+0000","last_active":"2023-06-08T03:50:29.312228+0000","last_peered":"2023-06-08T03:50:29.312228+0000","last_clean":"2023-06-08T03:50:29.312228+0000","last_became_active":"2023-06-08T03:50:24.277802+0000","last_became_peered":"2023-06-08T03:50:24.277802+0000","last_unstale":"2023-06-08T03:50:29.312228+0000","last_undegraded":"2023-06-08T03:50:29.312228+0000","last_fullsized":"2023-06-08T03:50:29.312228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789217+0000","last_change":"2023-06-08T03:50:24.699630+0000","last_active":"2023-06-08T03:50:29.789217+0000","last_peered":"2023-06-08T03:50:29.789217+0000","last_clean":"2023-06-08T03:50:29.789217+0000","last_became_active":"2023-06-08T03:50:24.699373+0000","last_became_peered":"2023-06-08T03:50:24.699373+0000","last_unstale":"2023-06-08T03:50:29.789217+0000","last_undegraded":"2023-06-08T03:50:29.789217+0000","last_fullsized":"2023-06-08T03:50:29.789217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:08:53.776951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789066+0000","last_change":"2023-06-08T03:50:28.923351+0000","last_active":"2023-06-08T03:50:29.789066+0000","last_peered":"2023-06-08T03:50:29.789066+0000","last_clean":"2023-06-08T03:50:29.789066+0000","last_became_active":"2023-06-08T03:50:28.922947+0000","last_became_peered":"2023-06-08T03:50:28.922947+0000","last_unstale":"2023-06-08T03:50:29.789066+0000","last_undegraded":"2023-06-08T03:50:29.789066+0000","last_fullsized":"2023-06-08T03:50:29.789066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:19:30.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,4],"acting":[2,1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790604+0000","last_change":"2023-06-08T03:50:24.276888+0000","last_active":"2023-06-08T03:50:29.790604+0000","last_peered":"2023-06-08T03:50:29.790604+0000","last_clean":"2023-06-08T03:50:29.790604+0000","last_became_active":"2023-06-08T03:50:24.276608+0000","last_became_peered":"2023-06-08T03:50:24.276608+0000","last_unstale":"2023-06-08T03:50:29.790604+0000","last_undegraded":"2023-06-08T03:50:29.790604+0000","last_fullsized":"2023-06-08T03:50:29.790604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:23.615413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307879+0000","last_change":"2023-06-08T03:50:24.690298+0000","last_active":"2023-06-08T03:50:29.307879+0000","last_peered":"2023-06-08T03:50:29.307879+0000","last_clean":"2023-06-08T03:50:29.307879+0000","last_became_active":"2023-06-08T03:50:24.689874+0000","last_became_peered":"2023-06-08T03:50:24.689874+0000","last_unstale":"2023-06-08T03:50:29.307879+0000","last_undegraded":"2023-06-08T03:50:29.307879+0000","last_fullsized":"2023-06-08T03:50:29.307879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309657+0000","last_change":"2023-06-08T03:50:28.925251+0000","last_active":"2023-06-08T03:50:29.309657+0000","last_peered":"2023-06-08T03:50:29.309657+0000","last_clean":"2023-06-08T03:50:29.309657+0000","last_became_active":"2023-06-08T03:50:28.923741+0000","last_became_peered":"2023-06-08T03:50:28.923741+0000","last_unstale":"2023-06-08T03:50:29.309657+0000","last_undegraded":"2023-06-08T03:50:29.309657+0000","last_fullsized":"2023-06-08T03:50:29.309657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:55:27.532871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7,1],"acting":[4,6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.285463+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.285152+0000","last_became_peered":"2023-06-08T03:50:24.285152+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308197+0000","last_change":"2023-06-08T03:50:24.689181+0000","last_active":"2023-06-08T03:50:29.308197+0000","last_peered":"2023-06-08T03:50:29.308197+0000","last_clean":"2023-06-08T03:50:29.308197+0000","last_became_active":"2023-06-08T03:50:24.688882+0000","last_became_peered":"2023-06-08T03:50:24.688882+0000","last_unstale":"2023-06-08T03:50:29.308197+0000","last_undegraded":"2023-06-08T03:50:29.308197+0000","last_fullsized":"2023-06-08T03:50:29.308197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:58:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308089+0000","last_change":"2023-06-08T03:50:28.919724+0000","last_active":"2023-06-08T03:50:29.308089+0000","last_peered":"2023-06-08T03:50:29.308089+0000","last_clean":"2023-06-08T03:50:29.308089+0000","last_became_active":"2023-06-08T03:50:28.919364+0000","last_became_peered":"2023-06-08T03:50:28.919364+0000","last_unstale":"2023-06-08T03:50:29.308089+0000","last_undegraded":"2023-06-08T03:50:29.308089+0000","last_fullsized":"2023-06-08T03:50:29.308089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:29.657933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4,3],"acting":[6,1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:34.204116+0000","last_change":"2023-06-08T03:50:24.284754+0000","last_active":"2023-06-08T03:50:34.204116+0000","last_peered":"2023-06-08T03:50:34.204116+0000","last_clean":"2023-06-08T03:50:34.204116+0000","last_became_active":"2023-06-08T03:50:24.281521+0000","last_became_peered":"2023-06-08T03:50:24.281521+0000","last_unstale":"2023-06-08T03:50:34.204116+0000","last_undegraded":"2023-06-08T03:50:34.204116+0000","last_fullsized":"2023-06-08T03:50:34.204116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:25.213919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:24.688053+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:24.687402+0000","last_became_peered":"2023-06-08T03:50:24.687402+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:02.669215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790804+0000","last_change":"2023-06-08T03:50:28.918485+0000","last_active":"2023-06-08T03:50:29.790804+0000","last_peered":"2023-06-08T03:50:29.790804+0000","last_clean":"2023-06-08T03:50:29.790804+0000","last_became_active":"2023-06-08T03:50:28.918103+0000","last_became_peered":"2023-06-08T03:50:28.918103+0000","last_unstale":"2023-06-08T03:50:29.790804+0000","last_undegraded":"2023-06-08T03:50:29.790804+0000","last_fullsized":"2023-06-08T03:50:29.790804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:26:43.524009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,6],"acting":[3,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789698+0000","last_change":"2023-06-08T03:50:24.287653+0000","last_active":"2023-06-08T03:50:29.789698+0000","last_peered":"2023-06-08T03:50:29.789698+0000","last_clean":"2023-06-08T03:50:29.789698+0000","last_became_active":"2023-06-08T03:50:24.287374+0000","last_became_peered":"2023-06-08T03:50:24.287374+0000","last_unstale":"2023-06-08T03:50:29.789698+0000","last_undegraded":"2023-06-08T03:50:29.789698+0000","last_fullsized":"2023-06-08T03:50:29.789698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:25:09.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308854+0000","last_change":"2023-06-08T03:50:24.693976+0000","last_active":"2023-06-08T03:50:29.308854+0000","last_peered":"2023-06-08T03:50:29.308854+0000","last_clean":"2023-06-08T03:50:29.308854+0000","last_became_active":"2023-06-08T03:50:24.693510+0000","last_became_peered":"2023-06-08T03:50:24.693510+0000","last_unstale":"2023-06-08T03:50:29.308854+0000","last_undegraded":"2023-06-08T03:50:29.308854+0000","last_fullsized":"2023-06-08T03:50:29.308854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:30.446775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310556+0000","last_change":"2023-06-08T03:50:28.925279+0000","last_active":"2023-06-08T03:50:29.310556+0000","last_peered":"2023-06-08T03:50:29.310556+0000","last_clean":"2023-06-08T03:50:29.310556+0000","last_became_active":"2023-06-08T03:50:28.923608+0000","last_became_peered":"2023-06-08T03:50:28.923608+0000","last_unstale":"2023-06-08T03:50:29.310556+0000","last_undegraded":"2023-06-08T03:50:29.310556+0000","last_fullsized":"2023-06-08T03:50:29.310556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:32.233874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5,7],"acting":[4,2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309210+0000","last_change":"2023-06-08T03:50:24.285047+0000","last_active":"2023-06-08T03:50:29.309210+0000","last_peered":"2023-06-08T03:50:29.309210+0000","last_clean":"2023-06-08T03:50:29.309210+0000","last_became_active":"2023-06-08T03:50:24.284768+0000","last_became_peered":"2023-06-08T03:50:24.284768+0000","last_unstale":"2023-06-08T03:50:29.309210+0000","last_undegraded":"2023-06-08T03:50:29.309210+0000","last_fullsized":"2023-06-08T03:50:29.309210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790899+0000","last_change":"2023-06-08T03:50:24.691967+0000","last_active":"2023-06-08T03:50:29.790899+0000","last_peered":"2023-06-08T03:50:29.790899+0000","last_clean":"2023-06-08T03:50:29.790899+0000","last_became_active":"2023-06-08T03:50:24.691699+0000","last_became_peered":"2023-06-08T03:50:24.691699+0000","last_unstale":"2023-06-08T03:50:29.790899+0000","last_undegraded":"2023-06-08T03:50:29.790899+0000","last_fullsized":"2023-06-08T03:50:29.790899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:40:30.017039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310163+0000","last_change":"2023-06-08T03:50:28.926277+0000","last_active":"2023-06-08T03:50:29.310163+0000","last_peered":"2023-06-08T03:50:29.310163+0000","last_clean":"2023-06-08T03:50:29.310163+0000","last_became_active":"2023-06-08T03:50:28.925921+0000","last_became_peered":"2023-06-08T03:50:28.925921+0000","last_unstale":"2023-06-08T03:50:29.310163+0000","last_undegraded":"2023-06-08T03:50:29.310163+0000","last_fullsized":"2023-06-08T03:50:29.310163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:32:27.673747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1,6],"acting":[4,5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789845+0000","last_change":"2023-06-08T03:50:24.283127+0000","last_active":"2023-06-08T03:50:29.789845+0000","last_peered":"2023-06-08T03:50:29.789845+0000","last_clean":"2023-06-08T03:50:29.789845+0000","last_became_active":"2023-06-08T03:50:24.282515+0000","last_became_peered":"2023-06-08T03:50:24.282515+0000","last_unstale":"2023-06-08T03:50:29.789845+0000","last_undegraded":"2023-06-08T03:50:29.789845+0000","last_fullsized":"2023-06-08T03:50:29.789845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:55.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311940+0000","last_change":"2023-06-08T03:50:24.687906+0000","last_active":"2023-06-08T03:50:29.311940+0000","last_peered":"2023-06-08T03:50:29.311940+0000","last_clean":"2023-06-08T03:50:29.311940+0000","last_became_active":"2023-06-08T03:50:24.687630+0000","last_became_peered":"2023-06-08T03:50:24.687630+0000","last_unstale":"2023-06-08T03:50:29.311940+0000","last_undegraded":"2023-06-08T03:50:29.311940+0000","last_fullsized":"2023-06-08T03:50:29.311940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:12.100669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790030+0000","last_change":"2023-06-08T03:50:28.918543+0000","last_active":"2023-06-08T03:50:29.790030+0000","last_peered":"2023-06-08T03:50:29.790030+0000","last_clean":"2023-06-08T03:50:29.790030+0000","last_became_active":"2023-06-08T03:50:28.918124+0000","last_became_peered":"2023-06-08T03:50:28.918124+0000","last_unstale":"2023-06-08T03:50:29.790030+0000","last_undegraded":"2023-06-08T03:50:29.790030+0000","last_fullsized":"2023-06-08T03:50:29.790030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:32:30.430868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4,0],"acting":[3,6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789271+0000","last_change":"2023-06-08T03:50:24.286716+0000","last_active":"2023-06-08T03:50:29.789271+0000","last_peered":"2023-06-08T03:50:29.789271+0000","last_clean":"2023-06-08T03:50:29.789271+0000","last_became_active":"2023-06-08T03:50:24.286096+0000","last_became_peered":"2023-06-08T03:50:24.286096+0000","last_unstale":"2023-06-08T03:50:29.789271+0000","last_undegraded":"2023-06-08T03:50:29.789271+0000","last_fullsized":"2023-06-08T03:50:29.789271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312273+0000","last_change":"2023-06-08T03:50:24.690535+0000","last_active":"2023-06-08T03:50:29.312273+0000","last_peered":"2023-06-08T03:50:29.312273+0000","last_clean":"2023-06-08T03:50:29.312273+0000","last_became_active":"2023-06-08T03:50:24.689649+0000","last_became_peered":"2023-06-08T03:50:24.689649+0000","last_unstale":"2023-06-08T03:50:29.312273+0000","last_undegraded":"2023-06-08T03:50:29.312273+0000","last_fullsized":"2023-06-08T03:50:29.312273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789697+0000","last_change":"2023-06-08T03:50:28.926759+0000","last_active":"2023-06-08T03:50:29.789697+0000","last_peered":"2023-06-08T03:50:29.789697+0000","last_clean":"2023-06-08T03:50:29.789697+0000","last_became_active":"2023-06-08T03:50:28.926391+0000","last_became_peered":"2023-06-08T03:50:28.926391+0000","last_unstale":"2023-06-08T03:50:29.789697+0000","last_undegraded":"2023-06-08T03:50:29.789697+0000","last_fullsized":"2023-06-08T03:50:29.789697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:28:31.793415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,7],"acting":[1,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.286445+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.286062+0000","last_became_peered":"2023-06-08T03:50:24.286062+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791098+0000","last_change":"2023-06-08T03:50:24.692233+0000","last_active":"2023-06-08T03:50:29.791098+0000","last_peered":"2023-06-08T03:50:29.791098+0000","last_clean":"2023-06-08T03:50:29.791098+0000","last_became_active":"2023-06-08T03:50:24.691584+0000","last_became_peered":"2023-06-08T03:50:24.691584+0000","last_unstale":"2023-06-08T03:50:29.791098+0000","last_undegraded":"2023-06-08T03:50:29.791098+0000","last_fullsized":"2023-06-08T03:50:29.791098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:43.387552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309967+0000","last_change":"2023-06-08T03:50:28.922207+0000","last_active":"2023-06-08T03:50:29.309967+0000","last_peered":"2023-06-08T03:50:29.309967+0000","last_clean":"2023-06-08T03:50:29.309967+0000","last_became_active":"2023-06-08T03:50:28.921809+0000","last_became_peered":"2023-06-08T03:50:28.921809+0000","last_unstale":"2023-06-08T03:50:29.309967+0000","last_undegraded":"2023-06-08T03:50:29.309967+0000","last_fullsized":"2023-06-08T03:50:29.309967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:23:47.506790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,0],"acting":[6,4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790800+0000","last_change":"2023-06-08T03:50:24.280168+0000","last_active":"2023-06-08T03:50:29.790800+0000","last_peered":"2023-06-08T03:50:29.790800+0000","last_clean":"2023-06-08T03:50:29.790800+0000","last_became_active":"2023-06-08T03:50:24.279947+0000","last_became_peered":"2023-06-08T03:50:24.279947+0000","last_unstale":"2023-06-08T03:50:29.790800+0000","last_undegraded":"2023-06-08T03:50:29.790800+0000","last_fullsized":"2023-06-08T03:50:29.790800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:11.091336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790420+0000","last_change":"2023-06-08T03:50:24.702952+0000","last_active":"2023-06-08T03:50:29.790420+0000","last_peered":"2023-06-08T03:50:29.790420+0000","last_clean":"2023-06-08T03:50:29.790420+0000","last_became_active":"2023-06-08T03:50:24.702213+0000","last_became_peered":"2023-06-08T03:50:24.702213+0000","last_unstale":"2023-06-08T03:50:29.790420+0000","last_undegraded":"2023-06-08T03:50:29.790420+0000","last_fullsized":"2023-06-08T03:50:29.790420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:57:59.540147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310770+0000","last_change":"2023-06-08T03:50:28.921768+0000","last_active":"2023-06-08T03:50:29.310770+0000","last_peered":"2023-06-08T03:50:29.310770+0000","last_clean":"2023-06-08T03:50:29.310770+0000","last_became_active":"2023-06-08T03:50:28.921477+0000","last_became_peered":"2023-06-08T03:50:28.921477+0000","last_unstale":"2023-06-08T03:50:29.310770+0000","last_undegraded":"2023-06-08T03:50:29.310770+0000","last_fullsized":"2023-06-08T03:50:29.310770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:06.300830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790697+0000","last_change":"2023-06-08T03:50:24.287775+0000","last_active":"2023-06-08T03:50:29.790697+0000","last_peered":"2023-06-08T03:50:29.790697+0000","last_clean":"2023-06-08T03:50:29.790697+0000","last_became_active":"2023-06-08T03:50:24.287136+0000","last_became_peered":"2023-06-08T03:50:24.287136+0000","last_unstale":"2023-06-08T03:50:29.790697+0000","last_undegraded":"2023-06-08T03:50:29.790697+0000","last_fullsized":"2023-06-08T03:50:29.790697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:17:17.207119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789501+0000","last_change":"2023-06-08T03:50:24.702234+0000","last_active":"2023-06-08T03:50:29.789501+0000","last_peered":"2023-06-08T03:50:29.789501+0000","last_clean":"2023-06-08T03:50:29.789501+0000","last_became_active":"2023-06-08T03:50:24.701881+0000","last_became_peered":"2023-06-08T03:50:24.701881+0000","last_unstale":"2023-06-08T03:50:29.789501+0000","last_undegraded":"2023-06-08T03:50:29.789501+0000","last_fullsized":"2023-06-08T03:50:29.789501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:52:37.709189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308237+0000","last_change":"2023-06-08T03:50:28.914578+0000","last_active":"2023-06-08T03:50:29.308237+0000","last_peered":"2023-06-08T03:50:29.308237+0000","last_clean":"2023-06-08T03:50:29.308237+0000","last_became_active":"2023-06-08T03:50:28.914064+0000","last_became_peered":"2023-06-08T03:50:28.914064+0000","last_unstale":"2023-06-08T03:50:29.308237+0000","last_undegraded":"2023-06-08T03:50:29.308237+0000","last_fullsized":"2023-06-08T03:50:29.308237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:52.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,7],"acting":[6,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.283466+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.283156+0000","last_became_peered":"2023-06-08T03:50:24.283156+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:25.016985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310840+0000","last_change":"2023-06-08T03:50:24.680665+0000","last_active":"2023-06-08T03:50:29.310840+0000","last_peered":"2023-06-08T03:50:29.310840+0000","last_clean":"2023-06-08T03:50:29.310840+0000","last_became_active":"2023-06-08T03:50:24.680424+0000","last_became_peered":"2023-06-08T03:50:24.680424+0000","last_unstale":"2023-06-08T03:50:29.310840+0000","last_undegraded":"2023-06-08T03:50:29.310840+0000","last_fullsized":"2023-06-08T03:50:29.310840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:24.624745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789596+0000","last_change":"2023-06-08T03:50:28.919899+0000","last_active":"2023-06-08T03:50:29.789596+0000","last_peered":"2023-06-08T03:50:29.789596+0000","last_clean":"2023-06-08T03:50:29.789596+0000","last_became_active":"2023-06-08T03:50:28.919515+0000","last_became_peered":"2023-06-08T03:50:28.919515+0000","last_unstale":"2023-06-08T03:50:29.789596+0000","last_undegraded":"2023-06-08T03:50:29.789596+0000","last_fullsized":"2023-06-08T03:50:29.789596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:49:21.771214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5,1],"acting":[7,0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789597+0000","last_change":"2023-06-08T03:50:24.282752+0000","last_active":"2023-06-08T03:50:29.789597+0000","last_peered":"2023-06-08T03:50:29.789597+0000","last_clean":"2023-06-08T03:50:29.789597+0000","last_became_active":"2023-06-08T03:50:24.282473+0000","last_became_peered":"2023-06-08T03:50:24.282473+0000","last_unstale":"2023-06-08T03:50:29.789597+0000","last_undegraded":"2023-06-08T03:50:29.789597+0000","last_fullsized":"2023-06-08T03:50:29.789597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:15:57.351189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591060+0000","last_change":"2023-06-08T03:50:24.691781+0000","last_active":"2023-06-08T03:50:29.591060+0000","last_peered":"2023-06-08T03:50:29.591060+0000","last_clean":"2023-06-08T03:50:29.591060+0000","last_became_active":"2023-06-08T03:50:24.690055+0000","last_became_peered":"2023-06-08T03:50:24.690055+0000","last_unstale":"2023-06-08T03:50:29.591060+0000","last_undegraded":"2023-06-08T03:50:29.591060+0000","last_fullsized":"2023-06-08T03:50:29.591060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311854+0000","last_change":"2023-06-08T03:50:28.927198+0000","last_active":"2023-06-08T03:50:29.311854+0000","last_peered":"2023-06-08T03:50:29.311854+0000","last_clean":"2023-06-08T03:50:29.311854+0000","last_became_active":"2023-06-08T03:50:28.926812+0000","last_became_peered":"2023-06-08T03:50:28.926812+0000","last_unstale":"2023-06-08T03:50:29.311854+0000","last_undegraded":"2023-06-08T03:50:29.311854+0000","last_fullsized":"2023-06-08T03:50:29.311854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4,2],"acting":[5,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789506+0000","last_change":"2023-06-08T03:50:24.287767+0000","last_active":"2023-06-08T03:50:29.789506+0000","last_peered":"2023-06-08T03:50:29.789506+0000","last_clean":"2023-06-08T03:50:29.789506+0000","last_became_active":"2023-06-08T03:50:24.287060+0000","last_became_peered":"2023-06-08T03:50:24.287060+0000","last_unstale":"2023-06-08T03:50:29.789506+0000","last_undegraded":"2023-06-08T03:50:29.789506+0000","last_fullsized":"2023-06-08T03:50:29.789506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:27.592926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790578+0000","last_change":"2023-06-08T03:50:24.700545+0000","last_active":"2023-06-08T03:50:29.790578+0000","last_peered":"2023-06-08T03:50:29.790578+0000","last_clean":"2023-06-08T03:50:29.790578+0000","last_became_active":"2023-06-08T03:50:24.700276+0000","last_became_peered":"2023-06-08T03:50:24.700276+0000","last_unstale":"2023-06-08T03:50:29.790578+0000","last_undegraded":"2023-06-08T03:50:29.790578+0000","last_fullsized":"2023-06-08T03:50:29.790578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:50.252975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791082+0000","last_change":"2023-06-08T03:50:28.920817+0000","last_active":"2023-06-08T03:50:29.791082+0000","last_peered":"2023-06-08T03:50:29.791082+0000","last_clean":"2023-06-08T03:50:29.791082+0000","last_became_active":"2023-06-08T03:50:28.920485+0000","last_became_peered":"2023-06-08T03:50:28.920485+0000","last_unstale":"2023-06-08T03:50:29.791082+0000","last_undegraded":"2023-06-08T03:50:29.791082+0000","last_fullsized":"2023-06-08T03:50:29.791082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:36.826227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,5],"acting":[1,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790009+0000","last_change":"2023-06-08T03:50:24.283331+0000","last_active":"2023-06-08T03:50:29.790009+0000","last_peered":"2023-06-08T03:50:29.790009+0000","last_clean":"2023-06-08T03:50:29.790009+0000","last_became_active":"2023-06-08T03:50:24.282856+0000","last_became_peered":"2023-06-08T03:50:24.282856+0000","last_unstale":"2023-06-08T03:50:29.790009+0000","last_undegraded":"2023-06-08T03:50:29.790009+0000","last_fullsized":"2023-06-08T03:50:29.790009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:52:45.337874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790917+0000","last_change":"2023-06-08T03:50:24.683795+0000","last_active":"2023-06-08T03:50:29.790917+0000","last_peered":"2023-06-08T03:50:29.790917+0000","last_clean":"2023-06-08T03:50:29.790917+0000","last_became_active":"2023-06-08T03:50:24.683510+0000","last_became_peered":"2023-06-08T03:50:24.683510+0000","last_unstale":"2023-06-08T03:50:29.790917+0000","last_undegraded":"2023-06-08T03:50:29.790917+0000","last_fullsized":"2023-06-08T03:50:29.790917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790162+0000","last_change":"2023-06-08T03:50:28.920202+0000","last_active":"2023-06-08T03:50:29.790162+0000","last_peered":"2023-06-08T03:50:29.790162+0000","last_clean":"2023-06-08T03:50:29.790162+0000","last_became_active":"2023-06-08T03:50:28.919960+0000","last_became_peered":"2023-06-08T03:50:28.919960+0000","last_unstale":"2023-06-08T03:50:29.790162+0000","last_undegraded":"2023-06-08T03:50:29.790162+0000","last_fullsized":"2023-06-08T03:50:29.790162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:44.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4,5],"acting":[3,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307873+0000","last_change":"2023-06-08T03:50:24.284333+0000","last_active":"2023-06-08T03:50:29.307873+0000","last_peered":"2023-06-08T03:50:29.307873+0000","last_clean":"2023-06-08T03:50:29.307873+0000","last_became_active":"2023-06-08T03:50:24.284039+0000","last_became_peered":"2023-06-08T03:50:24.284039+0000","last_unstale":"2023-06-08T03:50:29.307873+0000","last_undegraded":"2023-06-08T03:50:29.307873+0000","last_fullsized":"2023-06-08T03:50:29.307873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:37:13.194212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789927+0000","last_change":"2023-06-08T03:50:24.691444+0000","last_active":"2023-06-08T03:50:29.789927+0000","last_peered":"2023-06-08T03:50:29.789927+0000","last_clean":"2023-06-08T03:50:29.789927+0000","last_became_active":"2023-06-08T03:50:24.690971+0000","last_became_peered":"2023-06-08T03:50:24.690971+0000","last_unstale":"2023-06-08T03:50:29.789927+0000","last_undegraded":"2023-06-08T03:50:29.789927+0000","last_fullsized":"2023-06-08T03:50:29.789927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:38:23.297572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790270+0000","last_change":"2023-06-08T03:50:28.929890+0000","last_active":"2023-06-08T03:50:29.790270+0000","last_peered":"2023-06-08T03:50:29.790270+0000","last_clean":"2023-06-08T03:50:29.790270+0000","last_became_active":"2023-06-08T03:50:28.929496+0000","last_became_peered":"2023-06-08T03:50:28.929496+0000","last_unstale":"2023-06-08T03:50:29.790270+0000","last_undegraded":"2023-06-08T03:50:29.790270+0000","last_fullsized":"2023-06-08T03:50:29.790270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:21:51.680343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310379+0000","last_change":"2023-06-08T03:50:24.282724+0000","last_active":"2023-06-08T03:50:29.310379+0000","last_peered":"2023-06-08T03:50:29.310379+0000","last_clean":"2023-06-08T03:50:29.310379+0000","last_became_active":"2023-06-08T03:50:24.282451+0000","last_became_peered":"2023-06-08T03:50:24.282451+0000","last_unstale":"2023-06-08T03:50:29.310379+0000","last_undegraded":"2023-06-08T03:50:29.310379+0000","last_fullsized":"2023-06-08T03:50:29.310379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:44:39.775596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790625+0000","last_change":"2023-06-08T03:50:19.804887+0000","last_active":"2023-06-08T03:50:29.790625+0000","last_peered":"2023-06-08T03:50:29.790625+0000","last_clean":"2023-06-08T03:50:29.790625+0000","last_became_active":"2023-06-08T03:50:19.804546+0000","last_became_peered":"2023-06-08T03:50:19.804546+0000","last_unstale":"2023-06-08T03:50:29.790625+0000","last_undegraded":"2023-06-08T03:50:29.790625+0000","last_fullsized":"2023-06-08T03:50:29.790625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:34.859632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307694+0000","last_change":"2023-06-08T03:50:24.686843+0000","last_active":"2023-06-08T03:50:29.307694+0000","last_peered":"2023-06-08T03:50:29.307694+0000","last_clean":"2023-06-08T03:50:29.307694+0000","last_became_active":"2023-06-08T03:50:24.686587+0000","last_became_peered":"2023-06-08T03:50:24.686587+0000","last_unstale":"2023-06-08T03:50:29.307694+0000","last_undegraded":"2023-06-08T03:50:29.307694+0000","last_fullsized":"2023-06-08T03:50:29.307694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:36.573402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308557+0000","last_change":"2023-06-08T03:50:28.924976+0000","last_active":"2023-06-08T03:50:29.308557+0000","last_peered":"2023-06-08T03:50:29.308557+0000","last_clean":"2023-06-08T03:50:29.308557+0000","last_became_active":"2023-06-08T03:50:28.924335+0000","last_became_peered":"2023-06-08T03:50:28.924335+0000","last_unstale":"2023-06-08T03:50:29.308557+0000","last_undegraded":"2023-06-08T03:50:29.308557+0000","last_fullsized":"2023-06-08T03:50:29.308557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:47:44.099384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789646+0000","last_change":"2023-06-08T03:50:19.172271+0000","last_active":"2023-06-08T03:50:29.789646+0000","last_peered":"2023-06-08T03:50:29.789646+0000","last_clean":"2023-06-08T03:50:29.789646+0000","last_became_active":"2023-06-08T03:50:19.171917+0000","last_became_peered":"2023-06-08T03:50:19.171917+0000","last_unstale":"2023-06-08T03:50:29.789646+0000","last_undegraded":"2023-06-08T03:50:29.789646+0000","last_fullsized":"2023-06-08T03:50:29.789646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:01:56.709362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308387+0000","last_change":"2023-06-08T03:50:24.284253+0000","last_active":"2023-06-08T03:50:29.308387+0000","last_peered":"2023-06-08T03:50:29.308387+0000","last_clean":"2023-06-08T03:50:29.308387+0000","last_became_active":"2023-06-08T03:50:24.283958+0000","last_became_peered":"2023-06-08T03:50:24.283958+0000","last_unstale":"2023-06-08T03:50:29.308387+0000","last_undegraded":"2023-06-08T03:50:29.308387+0000","last_fullsized":"2023-06-08T03:50:29.308387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:02:34.479410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.690498+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.690242+0000","last_became_peered":"2023-06-08T03:50:24.690242+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:23:08.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790091+0000","last_change":"2023-06-08T03:50:28.921864+0000","last_active":"2023-06-08T03:50:29.790091+0000","last_peered":"2023-06-08T03:50:29.790091+0000","last_clean":"2023-06-08T03:50:29.790091+0000","last_became_active":"2023-06-08T03:50:28.921426+0000","last_became_peered":"2023-06-08T03:50:28.921426+0000","last_unstale":"2023-06-08T03:50:29.790091+0000","last_undegraded":"2023-06-08T03:50:29.790091+0000","last_fullsized":"2023-06-08T03:50:29.790091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:43:24.342082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,0],"acting":[3,7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309435+0000","last_change":"2023-06-08T03:50:19.171073+0000","last_active":"2023-06-08T03:50:29.309435+0000","last_peered":"2023-06-08T03:50:29.309435+0000","last_clean":"2023-06-08T03:50:29.309435+0000","last_became_active":"2023-06-08T03:50:19.170775+0000","last_became_peered":"2023-06-08T03:50:19.170775+0000","last_unstale":"2023-06-08T03:50:29.309435+0000","last_undegraded":"2023-06-08T03:50:29.309435+0000","last_fullsized":"2023-06-08T03:50:29.309435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:51:24.758705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791212+0000","last_change":"2023-06-08T03:50:19.167800+0000","last_active":"2023-06-08T03:50:29.791212+0000","last_peered":"2023-06-08T03:50:29.791212+0000","last_clean":"2023-06-08T03:50:29.791212+0000","last_became_active":"2023-06-08T03:50:19.167412+0000","last_became_peered":"2023-06-08T03:50:19.167412+0000","last_unstale":"2023-06-08T03:50:29.791212+0000","last_undegraded":"2023-06-08T03:50:29.791212+0000","last_fullsized":"2023-06-08T03:50:29.791212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:45:23.552252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309769+0000","last_change":"2023-06-08T03:50:24.280635+0000","last_active":"2023-06-08T03:50:29.309769+0000","last_peered":"2023-06-08T03:50:29.309769+0000","last_clean":"2023-06-08T03:50:29.309769+0000","last_became_active":"2023-06-08T03:50:24.280376+0000","last_became_peered":"2023-06-08T03:50:24.280376+0000","last_unstale":"2023-06-08T03:50:29.309769+0000","last_undegraded":"2023-06-08T03:50:29.309769+0000","last_fullsized":"2023-06-08T03:50:29.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:41.269585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312281+0000","last_change":"2023-06-08T03:50:24.685266+0000","last_active":"2023-06-08T03:50:29.312281+0000","last_peered":"2023-06-08T03:50:29.312281+0000","last_clean":"2023-06-08T03:50:29.312281+0000","last_became_active":"2023-06-08T03:50:24.684970+0000","last_became_peered":"2023-06-08T03:50:24.684970+0000","last_unstale":"2023-06-08T03:50:29.312281+0000","last_undegraded":"2023-06-08T03:50:29.312281+0000","last_fullsized":"2023-06-08T03:50:29.312281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789896+0000","last_change":"2023-06-08T03:50:28.926537+0000","last_active":"2023-06-08T03:50:29.789896+0000","last_peered":"2023-06-08T03:50:29.789896+0000","last_clean":"2023-06-08T03:50:29.789896+0000","last_became_active":"2023-06-08T03:50:28.926095+0000","last_became_peered":"2023-06-08T03:50:28.926095+0000","last_unstale":"2023-06-08T03:50:29.789896+0000","last_undegraded":"2023-06-08T03:50:29.789896+0000","last_fullsized":"2023-06-08T03:50:29.789896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:16:59.025169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,1],"acting":[7,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590877+0000","last_change":"2023-06-08T03:50:19.171834+0000","last_active":"2023-06-08T03:50:29.590877+0000","last_peered":"2023-06-08T03:50:29.590877+0000","last_clean":"2023-06-08T03:50:29.590877+0000","last_became_active":"2023-06-08T03:50:19.171541+0000","last_became_peered":"2023-06-08T03:50:19.171541+0000","last_unstale":"2023-06-08T03:50:29.590877+0000","last_undegraded":"2023-06-08T03:50:29.590877+0000","last_fullsized":"2023-06-08T03:50:29.590877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:42.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310150+0000","last_change":"2023-06-08T03:50:24.275533+0000","last_active":"2023-06-08T03:50:29.310150+0000","last_peered":"2023-06-08T03:50:29.310150+0000","last_clean":"2023-06-08T03:50:29.310150+0000","last_became_active":"2023-06-08T03:50:24.275086+0000","last_became_peered":"2023-06-08T03:50:24.275086+0000","last_unstale":"2023-06-08T03:50:29.310150+0000","last_undegraded":"2023-06-08T03:50:29.310150+0000","last_fullsized":"2023-06-08T03:50:29.310150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:49.514141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309093+0000","last_change":"2023-06-08T03:50:24.694692+0000","last_active":"2023-06-08T03:50:29.309093+0000","last_peered":"2023-06-08T03:50:29.309093+0000","last_clean":"2023-06-08T03:50:29.309093+0000","last_became_active":"2023-06-08T03:50:24.694404+0000","last_became_peered":"2023-06-08T03:50:24.694404+0000","last_unstale":"2023-06-08T03:50:29.309093+0000","last_undegraded":"2023-06-08T03:50:29.309093+0000","last_fullsized":"2023-06-08T03:50:29.309093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:06.873559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591398+0000","last_change":"2023-06-08T03:50:28.926999+0000","last_active":"2023-06-08T03:50:29.591398+0000","last_peered":"2023-06-08T03:50:29.591398+0000","last_clean":"2023-06-08T03:50:29.591398+0000","last_became_active":"2023-06-08T03:50:28.926614+0000","last_became_peered":"2023-06-08T03:50:28.926614+0000","last_unstale":"2023-06-08T03:50:29.591398+0000","last_undegraded":"2023-06-08T03:50:29.591398+0000","last_fullsized":"2023-06-08T03:50:29.591398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:45.910533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789240+0000","last_change":"2023-06-08T03:50:19.170601+0000","last_active":"2023-06-08T03:50:29.789240+0000","last_peered":"2023-06-08T03:50:29.789240+0000","last_clean":"2023-06-08T03:50:29.789240+0000","last_became_active":"2023-06-08T03:50:19.170344+0000","last_became_peered":"2023-06-08T03:50:19.170344+0000","last_unstale":"2023-06-08T03:50:29.789240+0000","last_undegraded":"2023-06-08T03:50:29.789240+0000","last_fullsized":"2023-06-08T03:50:29.789240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:35.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307871+0000","last_change":"2023-06-08T03:50:24.280451+0000","last_active":"2023-06-08T03:50:29.307871+0000","last_peered":"2023-06-08T03:50:29.307871+0000","last_clean":"2023-06-08T03:50:29.307871+0000","last_became_active":"2023-06-08T03:50:24.280118+0000","last_became_peered":"2023-06-08T03:50:24.280118+0000","last_unstale":"2023-06-08T03:50:29.307871+0000","last_undegraded":"2023-06-08T03:50:29.307871+0000","last_fullsized":"2023-06-08T03:50:29.307871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:18.146699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790813+0000","last_change":"2023-06-08T03:50:24.692660+0000","last_active":"2023-06-08T03:50:29.790813+0000","last_peered":"2023-06-08T03:50:29.790813+0000","last_clean":"2023-06-08T03:50:29.790813+0000","last_became_active":"2023-06-08T03:50:24.692373+0000","last_became_peered":"2023-06-08T03:50:24.692373+0000","last_unstale":"2023-06-08T03:50:29.790813+0000","last_undegraded":"2023-06-08T03:50:29.790813+0000","last_fullsized":"2023-06-08T03:50:29.790813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:14:48.949961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789804+0000","last_change":"2023-06-08T03:50:28.924634+0000","last_active":"2023-06-08T03:50:29.789804+0000","last_peered":"2023-06-08T03:50:29.789804+0000","last_clean":"2023-06-08T03:50:29.789804+0000","last_became_active":"2023-06-08T03:50:28.924227+0000","last_became_peered":"2023-06-08T03:50:28.924227+0000","last_unstale":"2023-06-08T03:50:29.789804+0000","last_undegraded":"2023-06-08T03:50:29.789804+0000","last_fullsized":"2023-06-08T03:50:29.789804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:24:05.389046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3,1],"acting":[7,4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789742+0000","last_change":"2023-06-08T03:50:19.805195+0000","last_active":"2023-06-08T03:50:29.789742+0000","last_peered":"2023-06-08T03:50:29.789742+0000","last_clean":"2023-06-08T03:50:29.789742+0000","last_became_active":"2023-06-08T03:50:19.804708+0000","last_became_peered":"2023-06-08T03:50:19.804708+0000","last_unstale":"2023-06-08T03:50:29.789742+0000","last_undegraded":"2023-06-08T03:50:29.789742+0000","last_fullsized":"2023-06-08T03:50:29.789742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:53:26.398016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591012+0000","last_change":"2023-06-08T03:50:24.281698+0000","last_active":"2023-06-08T03:50:29.591012+0000","last_peered":"2023-06-08T03:50:29.591012+0000","last_clean":"2023-06-08T03:50:29.591012+0000","last_became_active":"2023-06-08T03:50:24.281443+0000","last_became_peered":"2023-06-08T03:50:24.281443+0000","last_unstale":"2023-06-08T03:50:29.591012+0000","last_undegraded":"2023-06-08T03:50:29.591012+0000","last_fullsized":"2023-06-08T03:50:29.591012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:29.437443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309408+0000","last_change":"2023-06-08T03:50:24.693777+0000","last_active":"2023-06-08T03:50:29.309408+0000","last_peered":"2023-06-08T03:50:29.309408+0000","last_clean":"2023-06-08T03:50:29.309408+0000","last_became_active":"2023-06-08T03:50:24.693193+0000","last_became_peered":"2023-06-08T03:50:24.693193+0000","last_unstale":"2023-06-08T03:50:29.309408+0000","last_undegraded":"2023-06-08T03:50:29.309408+0000","last_fullsized":"2023-06-08T03:50:29.309408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591049+0000","last_change":"2023-06-08T03:50:28.926459+0000","last_active":"2023-06-08T03:50:29.591049+0000","last_peered":"2023-06-08T03:50:29.591049+0000","last_clean":"2023-06-08T03:50:29.591049+0000","last_became_active":"2023-06-08T03:50:28.926041+0000","last_became_peered":"2023-06-08T03:50:28.926041+0000","last_unstale":"2023-06-08T03:50:29.591049+0000","last_undegraded":"2023-06-08T03:50:29.591049+0000","last_fullsized":"2023-06-08T03:50:29.591049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:10.505175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,5],"acting":[0,7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:19.805012+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:19.804722+0000","last_became_peered":"2023-06-08T03:50:19.804722+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:43:32.916770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309261+0000","last_change":"2023-06-08T03:50:24.284390+0000","last_active":"2023-06-08T03:50:29.309261+0000","last_peered":"2023-06-08T03:50:29.309261+0000","last_clean":"2023-06-08T03:50:29.309261+0000","last_became_active":"2023-06-08T03:50:24.284129+0000","last_became_peered":"2023-06-08T03:50:24.284129+0000","last_unstale":"2023-06-08T03:50:29.309261+0000","last_undegraded":"2023-06-08T03:50:29.309261+0000","last_fullsized":"2023-06-08T03:50:29.309261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309328+0000","last_change":"2023-06-08T03:50:24.693789+0000","last_active":"2023-06-08T03:50:29.309328+0000","last_peered":"2023-06-08T03:50:29.309328+0000","last_clean":"2023-06-08T03:50:29.309328+0000","last_became_active":"2023-06-08T03:50:24.693205+0000","last_became_peered":"2023-06-08T03:50:24.693205+0000","last_unstale":"2023-06-08T03:50:29.309328+0000","last_undegraded":"2023-06-08T03:50:29.309328+0000","last_fullsized":"2023-06-08T03:50:29.309328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310499+0000","last_change":"2023-06-08T03:50:28.924218+0000","last_active":"2023-06-08T03:50:29.310499+0000","last_peered":"2023-06-08T03:50:29.310499+0000","last_clean":"2023-06-08T03:50:29.310499+0000","last_became_active":"2023-06-08T03:50:28.923473+0000","last_became_peered":"2023-06-08T03:50:28.923473+0000","last_unstale":"2023-06-08T03:50:29.310499+0000","last_undegraded":"2023-06-08T03:50:29.310499+0000","last_fullsized":"2023-06-08T03:50:29.310499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:11:10.428932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3,1],"acting":[4,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790922+0000","last_change":"2023-06-08T03:50:19.805107+0000","last_active":"2023-06-08T03:50:29.790922+0000","last_peered":"2023-06-08T03:50:29.790922+0000","last_clean":"2023-06-08T03:50:29.790922+0000","last_became_active":"2023-06-08T03:50:19.804754+0000","last_became_peered":"2023-06-08T03:50:19.804754+0000","last_unstale":"2023-06-08T03:50:29.790922+0000","last_undegraded":"2023-06-08T03:50:29.790922+0000","last_fullsized":"2023-06-08T03:50:29.790922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:13.049279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790682+0000","last_change":"2023-06-08T03:50:24.281311+0000","last_active":"2023-06-08T03:50:29.790682+0000","last_peered":"2023-06-08T03:50:29.790682+0000","last_clean":"2023-06-08T03:50:29.790682+0000","last_became_active":"2023-06-08T03:50:24.280858+0000","last_became_peered":"2023-06-08T03:50:24.280858+0000","last_unstale":"2023-06-08T03:50:29.790682+0000","last_undegraded":"2023-06-08T03:50:29.790682+0000","last_fullsized":"2023-06-08T03:50:29.790682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:01.113573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309301+0000","last_change":"2023-06-08T03:50:24.700010+0000","last_active":"2023-06-08T03:50:29.309301+0000","last_peered":"2023-06-08T03:50:29.309301+0000","last_clean":"2023-06-08T03:50:29.309301+0000","last_became_active":"2023-06-08T03:50:24.699801+0000","last_became_peered":"2023-06-08T03:50:24.699801+0000","last_unstale":"2023-06-08T03:50:29.309301+0000","last_undegraded":"2023-06-08T03:50:29.309301+0000","last_fullsized":"2023-06-08T03:50:29.309301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308391+0000","last_change":"2023-06-08T03:50:28.920073+0000","last_active":"2023-06-08T03:50:29.308391+0000","last_peered":"2023-06-08T03:50:29.308391+0000","last_clean":"2023-06-08T03:50:29.308391+0000","last_became_active":"2023-06-08T03:50:28.919601+0000","last_became_peered":"2023-06-08T03:50:28.919601+0000","last_unstale":"2023-06-08T03:50:29.308391+0000","last_undegraded":"2023-06-08T03:50:29.308391+0000","last_fullsized":"2023-06-08T03:50:29.308391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:38:02.965137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309419+0000","last_change":"2023-06-08T03:50:24.285734+0000","last_active":"2023-06-08T03:50:29.309419+0000","last_peered":"2023-06-08T03:50:29.309419+0000","last_clean":"2023-06-08T03:50:29.309419+0000","last_became_active":"2023-06-08T03:50:24.285421+0000","last_became_peered":"2023-06-08T03:50:24.285421+0000","last_unstale":"2023-06-08T03:50:29.309419+0000","last_undegraded":"2023-06-08T03:50:29.309419+0000","last_fullsized":"2023-06-08T03:50:29.309419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:13:35.050524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309586+0000","last_change":"2023-06-08T03:50:24.696432+0000","last_active":"2023-06-08T03:50:29.309586+0000","last_peered":"2023-06-08T03:50:29.309586+0000","last_clean":"2023-06-08T03:50:29.309586+0000","last_became_active":"2023-06-08T03:50:24.695650+0000","last_became_peered":"2023-06-08T03:50:24.695650+0000","last_unstale":"2023-06-08T03:50:29.309586+0000","last_undegraded":"2023-06-08T03:50:29.309586+0000","last_fullsized":"2023-06-08T03:50:29.309586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:33:18.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790496+0000","last_change":"2023-06-08T03:50:28.929940+0000","last_active":"2023-06-08T03:50:29.790496+0000","last_peered":"2023-06-08T03:50:29.790496+0000","last_clean":"2023-06-08T03:50:29.790496+0000","last_became_active":"2023-06-08T03:50:28.929641+0000","last_became_peered":"2023-06-08T03:50:28.929641+0000","last_unstale":"2023-06-08T03:50:29.790496+0000","last_undegraded":"2023-06-08T03:50:29.790496+0000","last_fullsized":"2023-06-08T03:50:29.790496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:40.205018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.285199+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.284917+0000","last_became_peered":"2023-06-08T03:50:24.284917+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790180+0000","last_change":"2023-06-08T03:50:24.680708+0000","last_active":"2023-06-08T03:50:29.790180+0000","last_peered":"2023-06-08T03:50:29.790180+0000","last_clean":"2023-06-08T03:50:29.790180+0000","last_became_active":"2023-06-08T03:50:24.680437+0000","last_became_peered":"2023-06-08T03:50:24.680437+0000","last_unstale":"2023-06-08T03:50:29.790180+0000","last_undegraded":"2023-06-08T03:50:29.790180+0000","last_fullsized":"2023-06-08T03:50:29.790180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:44.278509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789986+0000","last_change":"2023-06-08T03:50:28.922303+0000","last_active":"2023-06-08T03:50:29.789986+0000","last_peered":"2023-06-08T03:50:29.789986+0000","last_clean":"2023-06-08T03:50:29.789986+0000","last_became_active":"2023-06-08T03:50:28.921661+0000","last_became_peered":"2023-06-08T03:50:28.921661+0000","last_unstale":"2023-06-08T03:50:29.789986+0000","last_undegraded":"2023-06-08T03:50:29.789986+0000","last_fullsized":"2023-06-08T03:50:29.789986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:54.155089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,4],"acting":[2,7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790594+0000","last_change":"2023-06-08T03:50:24.287963+0000","last_active":"2023-06-08T03:50:29.790594+0000","last_peered":"2023-06-08T03:50:29.790594+0000","last_clean":"2023-06-08T03:50:29.790594+0000","last_became_active":"2023-06-08T03:50:24.287513+0000","last_became_peered":"2023-06-08T03:50:24.287513+0000","last_unstale":"2023-06-08T03:50:29.790594+0000","last_undegraded":"2023-06-08T03:50:29.790594+0000","last_fullsized":"2023-06-08T03:50:29.790594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:41:31.366674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789388+0000","last_change":"2023-06-08T03:50:24.694656+0000","last_active":"2023-06-08T03:50:29.789388+0000","last_peered":"2023-06-08T03:50:29.789388+0000","last_clean":"2023-06-08T03:50:29.789388+0000","last_became_active":"2023-06-08T03:50:24.694387+0000","last_became_peered":"2023-06-08T03:50:24.694387+0000","last_unstale":"2023-06-08T03:50:29.789388+0000","last_undegraded":"2023-06-08T03:50:29.789388+0000","last_fullsized":"2023-06-08T03:50:29.789388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:11:50.398160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789148+0000","last_change":"2023-06-08T03:50:28.926194+0000","last_active":"2023-06-08T03:50:29.789148+0000","last_peered":"2023-06-08T03:50:29.789148+0000","last_clean":"2023-06-08T03:50:29.789148+0000","last_became_active":"2023-06-08T03:50:28.925783+0000","last_became_peered":"2023-06-08T03:50:28.925783+0000","last_unstale":"2023-06-08T03:50:29.789148+0000","last_undegraded":"2023-06-08T03:50:29.789148+0000","last_fullsized":"2023-06-08T03:50:29.789148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:26.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5,7],"acting":[1,0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312099+0000","last_change":"2023-06-08T03:50:24.277779+0000","last_active":"2023-06-08T03:50:29.312099+0000","last_peered":"2023-06-08T03:50:29.312099+0000","last_clean":"2023-06-08T03:50:29.312099+0000","last_became_active":"2023-06-08T03:50:24.277493+0000","last_became_peered":"2023-06-08T03:50:24.277493+0000","last_unstale":"2023-06-08T03:50:29.312099+0000","last_undegraded":"2023-06-08T03:50:29.312099+0000","last_fullsized":"2023-06-08T03:50:29.312099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789292+0000","last_change":"2023-06-08T03:50:24.696150+0000","last_active":"2023-06-08T03:50:29.789292+0000","last_peered":"2023-06-08T03:50:29.789292+0000","last_clean":"2023-06-08T03:50:29.789292+0000","last_became_active":"2023-06-08T03:50:24.695605+0000","last_became_peered":"2023-06-08T03:50:24.695605+0000","last_unstale":"2023-06-08T03:50:29.789292+0000","last_undegraded":"2023-06-08T03:50:29.789292+0000","last_fullsized":"2023-06-08T03:50:29.789292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:28:06.091963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308122+0000","last_change":"2023-06-08T03:50:28.924072+0000","last_active":"2023-06-08T03:50:29.308122+0000","last_peered":"2023-06-08T03:50:29.308122+0000","last_clean":"2023-06-08T03:50:29.308122+0000","last_became_active":"2023-06-08T03:50:28.923155+0000","last_became_peered":"2023-06-08T03:50:28.923155+0000","last_unstale":"2023-06-08T03:50:29.308122+0000","last_undegraded":"2023-06-08T03:50:29.308122+0000","last_fullsized":"2023-06-08T03:50:29.308122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:00:25.670438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1,0],"acting":[4,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312115+0000","last_change":"2023-06-08T03:50:24.285764+0000","last_active":"2023-06-08T03:50:29.312115+0000","last_peered":"2023-06-08T03:50:29.312115+0000","last_clean":"2023-06-08T03:50:29.312115+0000","last_became_active":"2023-06-08T03:50:24.285477+0000","last_became_peered":"2023-06-08T03:50:24.285477+0000","last_unstale":"2023-06-08T03:50:29.312115+0000","last_undegraded":"2023-06-08T03:50:29.312115+0000","last_fullsized":"2023-06-08T03:50:29.312115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:41.022058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.788785+0000","last_change":"2023-06-08T03:50:24.679506+0000","last_active":"2023-06-08T03:50:29.788785+0000","last_peered":"2023-06-08T03:50:29.788785+0000","last_clean":"2023-06-08T03:50:29.788785+0000","last_became_active":"2023-06-08T03:50:24.679246+0000","last_became_peered":"2023-06-08T03:50:24.679246+0000","last_unstale":"2023-06-08T03:50:29.788785+0000","last_undegraded":"2023-06-08T03:50:29.788785+0000","last_fullsized":"2023-06-08T03:50:29.788785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:53:20.880498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789376+0000","last_change":"2023-06-08T03:50:24.690393+0000","last_active":"2023-06-08T03:50:29.789376+0000","last_peered":"2023-06-08T03:50:29.789376+0000","last_clean":"2023-06-08T03:50:29.789376+0000","last_became_active":"2023-06-08T03:50:24.690144+0000","last_became_peered":"2023-06-08T03:50:24.690144+0000","last_unstale":"2023-06-08T03:50:29.789376+0000","last_undegraded":"2023-06-08T03:50:29.789376+0000","last_fullsized":"2023-06-08T03:50:29.789376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309500+0000","last_change":"2023-06-08T03:50:24.693278+0000","last_active":"2023-06-08T03:50:29.309500+0000","last_peered":"2023-06-08T03:50:29.309500+0000","last_clean":"2023-06-08T03:50:29.309500+0000","last_became_active":"2023-06-08T03:50:24.692978+0000","last_became_peered":"2023-06-08T03:50:24.692978+0000","last_unstale":"2023-06-08T03:50:29.309500+0000","last_undegraded":"2023-06-08T03:50:29.309500+0000","last_fullsized":"2023-06-08T03:50:29.309500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591028+0000","last_change":"2023-06-08T03:50:24.700428+0000","last_active":"2023-06-08T03:50:29.591028+0000","last_peered":"2023-06-08T03:50:29.591028+0000","last_clean":"2023-06-08T03:50:29.591028+0000","last_became_active":"2023-06-08T03:50:24.700158+0000","last_became_peered":"2023-06-08T03:50:24.700158+0000","last_unstale":"2023-06-08T03:50:29.591028+0000","last_undegraded":"2023-06-08T03:50:29.591028+0000","last_fullsized":"2023-06-08T03:50:29.591028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310861+0000","last_change":"2023-06-08T03:50:24.690477+0000","last_active":"2023-06-08T03:50:29.310861+0000","last_peered":"2023-06-08T03:50:29.310861+0000","last_clean":"2023-06-08T03:50:29.310861+0000","last_became_active":"2023-06-08T03:50:24.690125+0000","last_became_peered":"2023-06-08T03:50:24.690125+0000","last_unstale":"2023-06-08T03:50:29.310861+0000","last_undegraded":"2023-06-08T03:50:29.310861+0000","last_fullsized":"2023-06-08T03:50:29.310861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:40.966384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311958+0000","last_change":"2023-06-08T03:50:24.682936+0000","last_active":"2023-06-08T03:50:29.311958+0000","last_peered":"2023-06-08T03:50:29.311958+0000","last_clean":"2023-06-08T03:50:29.311958+0000","last_became_active":"2023-06-08T03:50:24.681545+0000","last_became_peered":"2023-06-08T03:50:24.681545+0000","last_unstale":"2023-06-08T03:50:29.311958+0000","last_undegraded":"2023-06-08T03:50:29.311958+0000","last_fullsized":"2023-06-08T03:50:29.311958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":270336,"data_stored":109499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":110083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0630000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1689999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1340000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":109555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":113203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":880640,"data_stored":703675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":700359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-08T03:50:47.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-08T03:50:47.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 Processor -- start 2023-06-08T03:50:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 -- start start 2023-06-08T03:50:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd7015a640 0x7fdd700b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd7015aa10 0x7fdd700ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd700b0750 con 0x7fdd700ce930 2023-06-08T03:50:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700ceeb0 con 0x7fdd7015a640 2023-06-08T03:50:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.813+0000 7fdd75b62700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700b09a0 con 0x7fdd7015aa10 2023-06-08T03:50:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd7015a640 0x7fdd700b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd7015a640 0x7fdd700b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35328/0 (socket says 172.21.15.79:35328) 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 -- 172.21.15.79:0/2422074573 learned_addr learned my addr 172.21.15.79:0/2422074573 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6effd700 1 --2- 172.21.15.79:0/2422074573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd7015aa10 0x7fdd700ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 -- 172.21.15.79:0/2422074573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd7015aa10 msgr2=0x7fdd700ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 --2- 172.21.15.79:0/2422074573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd7015aa10 0x7fdd700ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 -- 172.21.15.79:0/2422074573 >> v1:172.21.15.156:6789/0 conn(0x7fdd700ce930 legacy=0x7fdd700af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 -- 172.21.15.79:0/2422074573 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd700b0c80 con 0x7fdd7015a640 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6f7fe700 1 --2- 172.21.15.79:0/2422074573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd700b1480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009920 tx=0x7fdd60009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46254aff5c657b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6e7fc700 1 -- 172.21.15.79:0/2422074573 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60004230 con 0x7fdd7015a640 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6e7fc700 1 -- 172.21.15.79:0/2422074573 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60004390 con 0x7fdd7015a640 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.814+0000 7fdd6e7fc700 1 -- 172.21.15.79:0/2422074573 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60009e70 con 0x7fdd7015a640 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 -- 172.21.15.79:0/2422074573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 msgr2=0x7fdd700b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 --2- 172.21.15.79:0/2422074573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd700b1480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009920 tx=0x7fdd60009c30 comp rx=0 tx=0).stop 2023-06-08T03:50:47.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 -- 172.21.15.79:0/2422074573 shutdown_connections 2023-06-08T03:50:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 --2- 172.21.15.79:0/2422074573 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd700b1480 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 --2- 172.21.15.79:0/2422074573 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd7015aa10 0x7fdd700ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 -- 172.21.15.79:0/2422074573 >> 172.21.15.79:0/2422074573 conn(0x7fdd7006a670 msgr2=0x7fdd70099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 -- 172.21.15.79:0/2422074573 shutdown_connections 2023-06-08T03:50:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.815+0000 7fdd75b62700 1 -- 172.21.15.79:0/2422074573 wait complete. 2023-06-08T03:50:47.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.816+0000 7fdd75b62700 1 Processor -- start 2023-06-08T03:50:47.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.816+0000 7fdd75b62700 1 -- start start 2023-06-08T03:50:47.824 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd75b62700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd700ce930 0x7fdd701581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd75b62700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd701e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd75b62700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd7015aa10 0x7fdd7006fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd75b62700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70159100 con 0x7fdd7015a640 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd75b62700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70158730 con 0x7fdd7015aa10 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd75b62700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700b4a00 con 0x7fdd700ce930 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd701e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.825 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd701e2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35334/0 (socket says 172.21.15.79:35334) 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 -- 172.21.15.79:0/1194020899 learned_addr learned my addr 172.21.15.79:0/1194020899 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6f7fe700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd700ce930 0x7fdd701581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6ffff700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd7015aa10 0x7fdd7006fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 -- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd700ce930 msgr2=0x7fdd701581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd700ce930 0x7fdd701581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 -- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd7015aa10 msgr2=0x7fdd7006fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd7015aa10 0x7fdd7006fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:47.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 -- 172.21.15.79:0/1194020899 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60009550 con 0x7fdd7015a640 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6f7fe700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd700ce930 0x7fdd701581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6ffff700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd7015aa10 0x7fdd7006fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.817+0000 7fdd6effd700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd701e2580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d800 tx=0x7fdd6400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.818+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640125f0 con 0x7fdd7015a640 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.818+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64012750 con 0x7fdd7015a640 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.818+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd700b3be0 con 0x7fdd7015a640 2023-06-08T03:50:47.827 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.818+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64012b00 con 0x7fdd7015a640 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.818+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd700b4420 con 0x7fdd7015a640 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.819+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd700d12b0 con 0x7fdd7015a640 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.819+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd64012c60 con 0x7fdd7015a640 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.819+0000 7fdd57fff700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd5c06ace0 0x7fdd5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.819+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdd64089b60 con 0x7fdd7015a640 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.820+0000 7fdd6f7fe700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd5c06ace0 0x7fdd5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:47.828 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.820+0000 7fdd6f7fe700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd5c06ace0 0x7fdd5c06d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009130 tx=0x7fdd60008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:47.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:47.824+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd64054920 con 0x7fdd7015a640 2023-06-08T03:50:48.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.008+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 --> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd700b0f60 con 0x7fdd5c06ace0 2023-06-08T03:50:48.023 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.019+0000 7fdd57fff700 1 -- 172.21.15.79:0/1194020899 <== mgr.4099 v2:172.21.15.156:6832/2537078349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367594 (secure 0 0 0) 0x7fdd600041b0 con 0x7fdd5c06ace0 2023-06-08T03:50:48.026 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:48.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd5c06ace0 msgr2=0x7fdd5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.028 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd5c06ace0 0x7fdd5c06d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009130 tx=0x7fdd60008030 comp rx=0 tx=0).stop 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 msgr2=0x7fdd701e2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd701e2580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400d800 tx=0x7fdd6400dbc0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 shutdown_connections 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd7015aa10 0x7fdd7006fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd5c06ace0 0x7fdd5c06d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd700ce930 0x7fdd701581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 --2- 172.21.15.79:0/1194020899 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7015a640 0x7fdd701e2580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.029 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 >> 172.21.15.79:0/1194020899 conn(0x7fdd7006a670 msgr2=0x7fdd700728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:48.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 shutdown_connections 2023-06-08T03:50:48.030 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.023+0000 7fdd75b62700 1 -- 172.21.15.79:0/1194020899 wait complete. 2023-06-08T03:50:48.030 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-08T03:50:48.039 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-06-08T03:50:47.875081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594960,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218920,"kb_used_data":3304,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755800,"statfs":{"total":773094113280,"available":772869939200,"internally_reserved":0,"allocated":3383296,"data_stored":2065192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003717"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312366+0000","last_change":"2023-06-08T03:50:24.695168+0000","last_active":"2023-06-08T03:50:29.312366+0000","last_peered":"2023-06-08T03:50:29.312366+0000","last_clean":"2023-06-08T03:50:29.312366+0000","last_became_active":"2023-06-08T03:50:24.694872+0000","last_became_peered":"2023-06-08T03:50:24.694872+0000","last_unstale":"2023-06-08T03:50:29.312366+0000","last_undegraded":"2023-06-08T03:50:29.312366+0000","last_fullsized":"2023-06-08T03:50:29.312366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791005+0000","last_change":"2023-06-08T03:50:24.687043+0000","last_active":"2023-06-08T03:50:29.791005+0000","last_peered":"2023-06-08T03:50:29.791005+0000","last_clean":"2023-06-08T03:50:29.791005+0000","last_became_active":"2023-06-08T03:50:24.686710+0000","last_became_peered":"2023-06-08T03:50:24.686710+0000","last_unstale":"2023-06-08T03:50:29.791005+0000","last_undegraded":"2023-06-08T03:50:29.791005+0000","last_fullsized":"2023-06-08T03:50:29.791005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591260+0000","last_change":"2023-06-08T03:50:24.686759+0000","last_active":"2023-06-08T03:50:29.591260+0000","last_peered":"2023-06-08T03:50:29.591260+0000","last_clean":"2023-06-08T03:50:29.591260+0000","last_became_active":"2023-06-08T03:50:24.686497+0000","last_became_peered":"2023-06-08T03:50:24.686497+0000","last_unstale":"2023-06-08T03:50:29.591260+0000","last_undegraded":"2023-06-08T03:50:29.591260+0000","last_fullsized":"2023-06-08T03:50:29.591260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:14.651224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591320+0000","last_change":"2023-06-08T03:50:24.691468+0000","last_active":"2023-06-08T03:50:29.591320+0000","last_peered":"2023-06-08T03:50:29.591320+0000","last_clean":"2023-06-08T03:50:29.591320+0000","last_became_active":"2023-06-08T03:50:24.691218+0000","last_became_peered":"2023-06-08T03:50:24.691218+0000","last_unstale":"2023-06-08T03:50:29.591320+0000","last_undegraded":"2023-06-08T03:50:29.591320+0000","last_fullsized":"2023-06-08T03:50:29.591320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309624+0000","last_change":"2023-06-08T03:50:24.685604+0000","last_active":"2023-06-08T03:50:29.309624+0000","last_peered":"2023-06-08T03:50:29.309624+0000","last_clean":"2023-06-08T03:50:29.309624+0000","last_became_active":"2023-06-08T03:50:24.685184+0000","last_became_peered":"2023-06-08T03:50:24.685184+0000","last_unstale":"2023-06-08T03:50:29.309624+0000","last_undegraded":"2023-06-08T03:50:29.309624+0000","last_fullsized":"2023-06-08T03:50:29.309624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:09:46.870511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789772+0000","last_change":"2023-06-08T03:50:24.698386+0000","last_active":"2023-06-08T03:50:29.789772+0000","last_peered":"2023-06-08T03:50:29.789772+0000","last_clean":"2023-06-08T03:50:29.789772+0000","last_became_active":"2023-06-08T03:50:24.698086+0000","last_became_peered":"2023-06-08T03:50:24.698086+0000","last_unstale":"2023-06-08T03:50:29.789772+0000","last_undegraded":"2023-06-08T03:50:29.789772+0000","last_fullsized":"2023-06-08T03:50:29.789772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:11.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790111+0000","last_change":"2023-06-08T03:50:24.693046+0000","last_active":"2023-06-08T03:50:29.790111+0000","last_peered":"2023-06-08T03:50:29.790111+0000","last_clean":"2023-06-08T03:50:29.790111+0000","last_became_active":"2023-06-08T03:50:24.692581+0000","last_became_peered":"2023-06-08T03:50:24.692581+0000","last_unstale":"2023-06-08T03:50:29.790111+0000","last_undegraded":"2023-06-08T03:50:29.790111+0000","last_fullsized":"2023-06-08T03:50:29.790111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:39.113365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790425+0000","last_change":"2023-06-08T03:50:24.689352+0000","last_active":"2023-06-08T03:50:29.790425+0000","last_peered":"2023-06-08T03:50:29.790425+0000","last_clean":"2023-06-08T03:50:29.790425+0000","last_became_active":"2023-06-08T03:50:24.688961+0000","last_became_peered":"2023-06-08T03:50:24.688961+0000","last_unstale":"2023-06-08T03:50:29.790425+0000","last_undegraded":"2023-06-08T03:50:29.790425+0000","last_fullsized":"2023-06-08T03:50:29.790425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:59:48.614865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310084+0000","last_change":"2023-06-08T03:50:24.682755+0000","last_active":"2023-06-08T03:50:29.310084+0000","last_peered":"2023-06-08T03:50:29.310084+0000","last_clean":"2023-06-08T03:50:29.310084+0000","last_became_active":"2023-06-08T03:50:24.682417+0000","last_became_peered":"2023-06-08T03:50:24.682417+0000","last_unstale":"2023-06-08T03:50:29.310084+0000","last_undegraded":"2023-06-08T03:50:29.310084+0000","last_fullsized":"2023-06-08T03:50:29.310084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:38:17.580156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791038+0000","last_change":"2023-06-08T03:50:24.697828+0000","last_active":"2023-06-08T03:50:29.791038+0000","last_peered":"2023-06-08T03:50:29.791038+0000","last_clean":"2023-06-08T03:50:29.791038+0000","last_became_active":"2023-06-08T03:50:24.697576+0000","last_became_peered":"2023-06-08T03:50:24.697576+0000","last_unstale":"2023-06-08T03:50:29.791038+0000","last_undegraded":"2023-06-08T03:50:29.791038+0000","last_fullsized":"2023-06-08T03:50:29.791038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789439+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.789439+0000","last_peered":"2023-06-08T03:50:29.789439+0000","last_clean":"2023-06-08T03:50:29.789439+0000","last_became_active":"2023-06-08T03:50:24.688197+0000","last_became_peered":"2023-06-08T03:50:24.688197+0000","last_unstale":"2023-06-08T03:50:29.789439+0000","last_undegraded":"2023-06-08T03:50:29.789439+0000","last_fullsized":"2023-06-08T03:50:29.789439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:04:27.557616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311848+0000","last_change":"2023-06-08T03:50:24.692867+0000","last_active":"2023-06-08T03:50:29.311848+0000","last_peered":"2023-06-08T03:50:29.311848+0000","last_clean":"2023-06-08T03:50:29.311848+0000","last_became_active":"2023-06-08T03:50:24.692613+0000","last_became_peered":"2023-06-08T03:50:24.692613+0000","last_unstale":"2023-06-08T03:50:29.311848+0000","last_undegraded":"2023-06-08T03:50:29.311848+0000","last_fullsized":"2023-06-08T03:50:29.311848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:02.122906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309156+0000","last_change":"2023-06-08T03:50:24.694092+0000","last_active":"2023-06-08T03:50:29.309156+0000","last_peered":"2023-06-08T03:50:29.309156+0000","last_clean":"2023-06-08T03:50:29.309156+0000","last_became_active":"2023-06-08T03:50:24.693783+0000","last_became_peered":"2023-06-08T03:50:24.693783+0000","last_unstale":"2023-06-08T03:50:29.309156+0000","last_undegraded":"2023-06-08T03:50:29.309156+0000","last_fullsized":"2023-06-08T03:50:29.309156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790036+0000","last_change":"2023-06-08T03:50:24.702721+0000","last_active":"2023-06-08T03:50:29.790036+0000","last_peered":"2023-06-08T03:50:29.790036+0000","last_clean":"2023-06-08T03:50:29.790036+0000","last_became_active":"2023-06-08T03:50:24.702438+0000","last_became_peered":"2023-06-08T03:50:24.702438+0000","last_unstale":"2023-06-08T03:50:29.790036+0000","last_undegraded":"2023-06-08T03:50:29.790036+0000","last_fullsized":"2023-06-08T03:50:29.790036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:59:10.518497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.692560+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.692035+0000","last_became_peered":"2023-06-08T03:50:24.692035+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790991+0000","last_change":"2023-06-08T03:50:24.698511+0000","last_active":"2023-06-08T03:50:29.790991+0000","last_peered":"2023-06-08T03:50:29.790991+0000","last_clean":"2023-06-08T03:50:29.790991+0000","last_became_active":"2023-06-08T03:50:24.698281+0000","last_became_peered":"2023-06-08T03:50:24.698281+0000","last_unstale":"2023-06-08T03:50:29.790991+0000","last_undegraded":"2023-06-08T03:50:29.790991+0000","last_fullsized":"2023-06-08T03:50:29.790991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:27:46.498562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789941+0000","last_change":"2023-06-08T03:50:24.695969+0000","last_active":"2023-06-08T03:50:29.789941+0000","last_peered":"2023-06-08T03:50:29.789941+0000","last_clean":"2023-06-08T03:50:29.789941+0000","last_became_active":"2023-06-08T03:50:24.695711+0000","last_became_peered":"2023-06-08T03:50:24.695711+0000","last_unstale":"2023-06-08T03:50:29.789941+0000","last_undegraded":"2023-06-08T03:50:29.789941+0000","last_fullsized":"2023-06-08T03:50:29.789941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:45:45.278948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591321+0000","last_change":"2023-06-08T03:50:24.690923+0000","last_active":"2023-06-08T03:50:29.591321+0000","last_peered":"2023-06-08T03:50:29.591321+0000","last_clean":"2023-06-08T03:50:29.591321+0000","last_became_active":"2023-06-08T03:50:24.690667+0000","last_became_peered":"2023-06-08T03:50:24.690667+0000","last_unstale":"2023-06-08T03:50:29.591321+0000","last_undegraded":"2023-06-08T03:50:29.591321+0000","last_fullsized":"2023-06-08T03:50:29.591321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311961+0000","last_change":"2023-06-08T03:50:24.701274+0000","last_active":"2023-06-08T03:50:29.311961+0000","last_peered":"2023-06-08T03:50:29.311961+0000","last_clean":"2023-06-08T03:50:29.311961+0000","last_became_active":"2023-06-08T03:50:24.700991+0000","last_became_peered":"2023-06-08T03:50:24.700991+0000","last_unstale":"2023-06-08T03:50:29.311961+0000","last_undegraded":"2023-06-08T03:50:29.311961+0000","last_fullsized":"2023-06-08T03:50:29.311961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:23.142584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789321+0000","last_change":"2023-06-08T03:50:24.690034+0000","last_active":"2023-06-08T03:50:29.789321+0000","last_peered":"2023-06-08T03:50:29.789321+0000","last_clean":"2023-06-08T03:50:29.789321+0000","last_became_active":"2023-06-08T03:50:24.689524+0000","last_became_peered":"2023-06-08T03:50:24.689524+0000","last_unstale":"2023-06-08T03:50:29.789321+0000","last_undegraded":"2023-06-08T03:50:29.789321+0000","last_fullsized":"2023-06-08T03:50:29.789321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790401+0000","last_change":"2023-06-08T03:50:24.702598+0000","last_active":"2023-06-08T03:50:29.790401+0000","last_peered":"2023-06-08T03:50:29.790401+0000","last_clean":"2023-06-08T03:50:29.790401+0000","last_became_active":"2023-06-08T03:50:24.702337+0000","last_became_peered":"2023-06-08T03:50:24.702337+0000","last_unstale":"2023-06-08T03:50:29.790401+0000","last_undegraded":"2023-06-08T03:50:29.790401+0000","last_fullsized":"2023-06-08T03:50:29.790401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:29:18.553774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789291+0000","last_change":"2023-06-08T03:50:24.689664+0000","last_active":"2023-06-08T03:50:29.789291+0000","last_peered":"2023-06-08T03:50:29.789291+0000","last_clean":"2023-06-08T03:50:29.789291+0000","last_became_active":"2023-06-08T03:50:24.689386+0000","last_became_peered":"2023-06-08T03:50:24.689386+0000","last_unstale":"2023-06-08T03:50:29.789291+0000","last_undegraded":"2023-06-08T03:50:29.789291+0000","last_fullsized":"2023-06-08T03:50:29.789291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:23:34.826399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308547+0000","last_change":"2023-06-08T03:50:24.690551+0000","last_active":"2023-06-08T03:50:29.308547+0000","last_peered":"2023-06-08T03:50:29.308547+0000","last_clean":"2023-06-08T03:50:29.308547+0000","last_became_active":"2023-06-08T03:50:24.689459+0000","last_became_peered":"2023-06-08T03:50:24.689459+0000","last_unstale":"2023-06-08T03:50:29.308547+0000","last_undegraded":"2023-06-08T03:50:29.308547+0000","last_fullsized":"2023-06-08T03:50:29.308547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:21:13.898688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789394+0000","last_change":"2023-06-08T03:50:24.702787+0000","last_active":"2023-06-08T03:50:29.789394+0000","last_peered":"2023-06-08T03:50:29.789394+0000","last_clean":"2023-06-08T03:50:29.789394+0000","last_became_active":"2023-06-08T03:50:24.702390+0000","last_became_peered":"2023-06-08T03:50:24.702390+0000","last_unstale":"2023-06-08T03:50:29.789394+0000","last_undegraded":"2023-06-08T03:50:29.789394+0000","last_fullsized":"2023-06-08T03:50:29.789394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:24:57.523322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310288+0000","last_change":"2023-06-08T03:50:24.689083+0000","last_active":"2023-06-08T03:50:29.310288+0000","last_peered":"2023-06-08T03:50:29.310288+0000","last_clean":"2023-06-08T03:50:29.310288+0000","last_became_active":"2023-06-08T03:50:24.688803+0000","last_became_peered":"2023-06-08T03:50:24.688803+0000","last_unstale":"2023-06-08T03:50:29.310288+0000","last_undegraded":"2023-06-08T03:50:29.310288+0000","last_fullsized":"2023-06-08T03:50:29.310288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:59:04.642632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789549+0000","last_change":"2023-06-08T03:50:24.689752+0000","last_active":"2023-06-08T03:50:29.789549+0000","last_peered":"2023-06-08T03:50:29.789549+0000","last_clean":"2023-06-08T03:50:29.789549+0000","last_became_active":"2023-06-08T03:50:24.689199+0000","last_became_peered":"2023-06-08T03:50:24.689199+0000","last_unstale":"2023-06-08T03:50:29.789549+0000","last_undegraded":"2023-06-08T03:50:29.789549+0000","last_fullsized":"2023-06-08T03:50:29.789549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:06:49.219275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790465+0000","last_change":"2023-06-08T03:50:24.699485+0000","last_active":"2023-06-08T03:50:29.790465+0000","last_peered":"2023-06-08T03:50:29.790465+0000","last_clean":"2023-06-08T03:50:29.790465+0000","last_became_active":"2023-06-08T03:50:24.699155+0000","last_became_peered":"2023-06-08T03:50:24.699155+0000","last_unstale":"2023-06-08T03:50:29.790465+0000","last_undegraded":"2023-06-08T03:50:29.790465+0000","last_fullsized":"2023-06-08T03:50:29.790465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:09:23.122980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790702+0000","last_change":"2023-06-08T03:50:28.917312+0000","last_active":"2023-06-08T03:50:29.790702+0000","last_peered":"2023-06-08T03:50:29.790702+0000","last_clean":"2023-06-08T03:50:29.790702+0000","last_became_active":"2023-06-08T03:50:28.916901+0000","last_became_peered":"2023-06-08T03:50:28.916901+0000","last_unstale":"2023-06-08T03:50:29.790702+0000","last_undegraded":"2023-06-08T03:50:29.790702+0000","last_fullsized":"2023-06-08T03:50:29.790702+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6,0],"acting":[2,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789907+0000","last_change":"2023-06-08T03:50:24.285058+0000","last_active":"2023-06-08T03:50:29.789907+0000","last_peered":"2023-06-08T03:50:29.789907+0000","last_clean":"2023-06-08T03:50:29.789907+0000","last_became_active":"2023-06-08T03:50:24.284770+0000","last_became_peered":"2023-06-08T03:50:24.284770+0000","last_unstale":"2023-06-08T03:50:29.789907+0000","last_undegraded":"2023-06-08T03:50:29.789907+0000","last_fullsized":"2023-06-08T03:50:29.789907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:16:20.267305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789601+0000","last_change":"2023-06-08T03:50:24.698385+0000","last_active":"2023-06-08T03:50:29.789601+0000","last_peered":"2023-06-08T03:50:29.789601+0000","last_clean":"2023-06-08T03:50:29.789601+0000","last_became_active":"2023-06-08T03:50:24.697966+0000","last_became_peered":"2023-06-08T03:50:24.697966+0000","last_unstale":"2023-06-08T03:50:29.789601+0000","last_undegraded":"2023-06-08T03:50:29.789601+0000","last_fullsized":"2023-06-08T03:50:29.789601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:45:20.947165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791158+0000","last_change":"2023-06-08T03:50:28.922104+0000","last_active":"2023-06-08T03:50:29.791158+0000","last_peered":"2023-06-08T03:50:29.791158+0000","last_clean":"2023-06-08T03:50:29.791158+0000","last_became_active":"2023-06-08T03:50:28.921721+0000","last_became_peered":"2023-06-08T03:50:28.921721+0000","last_unstale":"2023-06-08T03:50:29.791158+0000","last_undegraded":"2023-06-08T03:50:29.791158+0000","last_fullsized":"2023-06-08T03:50:29.791158+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:28.031781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,5],"acting":[2,7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308030+0000","last_change":"2023-06-08T03:50:24.286192+0000","last_active":"2023-06-08T03:50:29.308030+0000","last_peered":"2023-06-08T03:50:29.308030+0000","last_clean":"2023-06-08T03:50:29.308030+0000","last_became_active":"2023-06-08T03:50:24.285939+0000","last_became_peered":"2023-06-08T03:50:24.285939+0000","last_unstale":"2023-06-08T03:50:29.308030+0000","last_undegraded":"2023-06-08T03:50:29.308030+0000","last_fullsized":"2023-06-08T03:50:29.308030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:32.185279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308525+0000","last_change":"2023-06-08T03:50:24.688443+0000","last_active":"2023-06-08T03:50:29.308525+0000","last_peered":"2023-06-08T03:50:29.308525+0000","last_clean":"2023-06-08T03:50:29.308525+0000","last_became_active":"2023-06-08T03:50:24.688036+0000","last_became_peered":"2023-06-08T03:50:24.688036+0000","last_unstale":"2023-06-08T03:50:29.308525+0000","last_undegraded":"2023-06-08T03:50:29.308525+0000","last_fullsized":"2023-06-08T03:50:29.308525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:23:40.833244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309722+0000","last_change":"2023-06-08T03:50:28.921813+0000","last_active":"2023-06-08T03:50:29.309722+0000","last_peered":"2023-06-08T03:50:29.309722+0000","last_clean":"2023-06-08T03:50:29.309722+0000","last_became_active":"2023-06-08T03:50:28.921392+0000","last_became_peered":"2023-06-08T03:50:28.921392+0000","last_unstale":"2023-06-08T03:50:29.309722+0000","last_undegraded":"2023-06-08T03:50:29.309722+0000","last_fullsized":"2023-06-08T03:50:29.309722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:51:18.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2,7],"acting":[6,3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310612+0000","last_change":"2023-06-08T03:50:24.281038+0000","last_active":"2023-06-08T03:50:29.310612+0000","last_peered":"2023-06-08T03:50:29.310612+0000","last_clean":"2023-06-08T03:50:29.310612+0000","last_became_active":"2023-06-08T03:50:24.280771+0000","last_became_peered":"2023-06-08T03:50:24.280771+0000","last_unstale":"2023-06-08T03:50:29.310612+0000","last_undegraded":"2023-06-08T03:50:29.310612+0000","last_fullsized":"2023-06-08T03:50:29.310612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:43.269177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790403+0000","last_change":"2023-06-08T03:50:24.686123+0000","last_active":"2023-06-08T03:50:29.790403+0000","last_peered":"2023-06-08T03:50:29.790403+0000","last_clean":"2023-06-08T03:50:29.790403+0000","last_became_active":"2023-06-08T03:50:24.685951+0000","last_became_peered":"2023-06-08T03:50:24.685951+0000","last_unstale":"2023-06-08T03:50:29.790403+0000","last_undegraded":"2023-06-08T03:50:29.790403+0000","last_fullsized":"2023-06-08T03:50:29.790403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:44:16.167507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789665+0000","last_change":"2023-06-08T03:50:28.929521+0000","last_active":"2023-06-08T03:50:29.789665+0000","last_peered":"2023-06-08T03:50:29.789665+0000","last_clean":"2023-06-08T03:50:29.789665+0000","last_became_active":"2023-06-08T03:50:28.929194+0000","last_became_peered":"2023-06-08T03:50:28.929194+0000","last_unstale":"2023-06-08T03:50:29.789665+0000","last_undegraded":"2023-06-08T03:50:29.789665+0000","last_fullsized":"2023-06-08T03:50:29.789665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:19:17.680617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,5],"acting":[3,2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312228+0000","last_change":"2023-06-08T03:50:24.278075+0000","last_active":"2023-06-08T03:50:29.312228+0000","last_peered":"2023-06-08T03:50:29.312228+0000","last_clean":"2023-06-08T03:50:29.312228+0000","last_became_active":"2023-06-08T03:50:24.277802+0000","last_became_peered":"2023-06-08T03:50:24.277802+0000","last_unstale":"2023-06-08T03:50:29.312228+0000","last_undegraded":"2023-06-08T03:50:29.312228+0000","last_fullsized":"2023-06-08T03:50:29.312228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789217+0000","last_change":"2023-06-08T03:50:24.699630+0000","last_active":"2023-06-08T03:50:29.789217+0000","last_peered":"2023-06-08T03:50:29.789217+0000","last_clean":"2023-06-08T03:50:29.789217+0000","last_became_active":"2023-06-08T03:50:24.699373+0000","last_became_peered":"2023-06-08T03:50:24.699373+0000","last_unstale":"2023-06-08T03:50:29.789217+0000","last_undegraded":"2023-06-08T03:50:29.789217+0000","last_fullsized":"2023-06-08T03:50:29.789217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:08:53.776951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789066+0000","last_change":"2023-06-08T03:50:28.923351+0000","last_active":"2023-06-08T03:50:29.789066+0000","last_peered":"2023-06-08T03:50:29.789066+0000","last_clean":"2023-06-08T03:50:29.789066+0000","last_became_active":"2023-06-08T03:50:28.922947+0000","last_became_peered":"2023-06-08T03:50:28.922947+0000","last_unstale":"2023-06-08T03:50:29.789066+0000","last_undegraded":"2023-06-08T03:50:29.789066+0000","last_fullsized":"2023-06-08T03:50:29.789066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:19:30.773676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,4],"acting":[2,1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790604+0000","last_change":"2023-06-08T03:50:24.276888+0000","last_active":"2023-06-08T03:50:29.790604+0000","last_peered":"2023-06-08T03:50:29.790604+0000","last_clean":"2023-06-08T03:50:29.790604+0000","last_became_active":"2023-06-08T03:50:24.276608+0000","last_became_peered":"2023-06-08T03:50:24.276608+0000","last_unstale":"2023-06-08T03:50:29.790604+0000","last_undegraded":"2023-06-08T03:50:29.790604+0000","last_fullsized":"2023-06-08T03:50:29.790604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:23.615413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307879+0000","last_change":"2023-06-08T03:50:24.690298+0000","last_active":"2023-06-08T03:50:29.307879+0000","last_peered":"2023-06-08T03:50:29.307879+0000","last_clean":"2023-06-08T03:50:29.307879+0000","last_became_active":"2023-06-08T03:50:24.689874+0000","last_became_peered":"2023-06-08T03:50:24.689874+0000","last_unstale":"2023-06-08T03:50:29.307879+0000","last_undegraded":"2023-06-08T03:50:29.307879+0000","last_fullsized":"2023-06-08T03:50:29.307879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:04:29.772034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309657+0000","last_change":"2023-06-08T03:50:28.925251+0000","last_active":"2023-06-08T03:50:29.309657+0000","last_peered":"2023-06-08T03:50:29.309657+0000","last_clean":"2023-06-08T03:50:29.309657+0000","last_became_active":"2023-06-08T03:50:28.923741+0000","last_became_peered":"2023-06-08T03:50:28.923741+0000","last_unstale":"2023-06-08T03:50:29.309657+0000","last_undegraded":"2023-06-08T03:50:29.309657+0000","last_fullsized":"2023-06-08T03:50:29.309657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:55:27.532871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7,1],"acting":[4,6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.285463+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.285152+0000","last_became_peered":"2023-06-08T03:50:24.285152+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308197+0000","last_change":"2023-06-08T03:50:24.689181+0000","last_active":"2023-06-08T03:50:29.308197+0000","last_peered":"2023-06-08T03:50:29.308197+0000","last_clean":"2023-06-08T03:50:29.308197+0000","last_became_active":"2023-06-08T03:50:24.688882+0000","last_became_peered":"2023-06-08T03:50:24.688882+0000","last_unstale":"2023-06-08T03:50:29.308197+0000","last_undegraded":"2023-06-08T03:50:29.308197+0000","last_fullsized":"2023-06-08T03:50:29.308197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:58:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308089+0000","last_change":"2023-06-08T03:50:28.919724+0000","last_active":"2023-06-08T03:50:29.308089+0000","last_peered":"2023-06-08T03:50:29.308089+0000","last_clean":"2023-06-08T03:50:29.308089+0000","last_became_active":"2023-06-08T03:50:28.919364+0000","last_became_peered":"2023-06-08T03:50:28.919364+0000","last_unstale":"2023-06-08T03:50:29.308089+0000","last_undegraded":"2023-06-08T03:50:29.308089+0000","last_fullsized":"2023-06-08T03:50:29.308089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:29.657933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4,3],"acting":[6,1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:34.204116+0000","last_change":"2023-06-08T03:50:24.284754+0000","last_active":"2023-06-08T03:50:34.204116+0000","last_peered":"2023-06-08T03:50:34.204116+0000","last_clean":"2023-06-08T03:50:34.204116+0000","last_became_active":"2023-06-08T03:50:24.281521+0000","last_became_peered":"2023-06-08T03:50:24.281521+0000","last_unstale":"2023-06-08T03:50:34.204116+0000","last_undegraded":"2023-06-08T03:50:34.204116+0000","last_fullsized":"2023-06-08T03:50:34.204116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:25.213919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:24.688053+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:24.687402+0000","last_became_peered":"2023-06-08T03:50:24.687402+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:02.669215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790804+0000","last_change":"2023-06-08T03:50:28.918485+0000","last_active":"2023-06-08T03:50:29.790804+0000","last_peered":"2023-06-08T03:50:29.790804+0000","last_clean":"2023-06-08T03:50:29.790804+0000","last_became_active":"2023-06-08T03:50:28.918103+0000","last_became_peered":"2023-06-08T03:50:28.918103+0000","last_unstale":"2023-06-08T03:50:29.790804+0000","last_undegraded":"2023-06-08T03:50:29.790804+0000","last_fullsized":"2023-06-08T03:50:29.790804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:26:43.524009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,6],"acting":[3,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789698+0000","last_change":"2023-06-08T03:50:24.287653+0000","last_active":"2023-06-08T03:50:29.789698+0000","last_peered":"2023-06-08T03:50:29.789698+0000","last_clean":"2023-06-08T03:50:29.789698+0000","last_became_active":"2023-06-08T03:50:24.287374+0000","last_became_peered":"2023-06-08T03:50:24.287374+0000","last_unstale":"2023-06-08T03:50:29.789698+0000","last_undegraded":"2023-06-08T03:50:29.789698+0000","last_fullsized":"2023-06-08T03:50:29.789698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:25:09.086331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308854+0000","last_change":"2023-06-08T03:50:24.693976+0000","last_active":"2023-06-08T03:50:29.308854+0000","last_peered":"2023-06-08T03:50:29.308854+0000","last_clean":"2023-06-08T03:50:29.308854+0000","last_became_active":"2023-06-08T03:50:24.693510+0000","last_became_peered":"2023-06-08T03:50:24.693510+0000","last_unstale":"2023-06-08T03:50:29.308854+0000","last_undegraded":"2023-06-08T03:50:29.308854+0000","last_fullsized":"2023-06-08T03:50:29.308854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:30.446775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310556+0000","last_change":"2023-06-08T03:50:28.925279+0000","last_active":"2023-06-08T03:50:29.310556+0000","last_peered":"2023-06-08T03:50:29.310556+0000","last_clean":"2023-06-08T03:50:29.310556+0000","last_became_active":"2023-06-08T03:50:28.923608+0000","last_became_peered":"2023-06-08T03:50:28.923608+0000","last_unstale":"2023-06-08T03:50:29.310556+0000","last_undegraded":"2023-06-08T03:50:29.310556+0000","last_fullsized":"2023-06-08T03:50:29.310556+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:32.233874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5,7],"acting":[4,2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309210+0000","last_change":"2023-06-08T03:50:24.285047+0000","last_active":"2023-06-08T03:50:29.309210+0000","last_peered":"2023-06-08T03:50:29.309210+0000","last_clean":"2023-06-08T03:50:29.309210+0000","last_became_active":"2023-06-08T03:50:24.284768+0000","last_became_peered":"2023-06-08T03:50:24.284768+0000","last_unstale":"2023-06-08T03:50:29.309210+0000","last_undegraded":"2023-06-08T03:50:29.309210+0000","last_fullsized":"2023-06-08T03:50:29.309210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:25:43.793098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790899+0000","last_change":"2023-06-08T03:50:24.691967+0000","last_active":"2023-06-08T03:50:29.790899+0000","last_peered":"2023-06-08T03:50:29.790899+0000","last_clean":"2023-06-08T03:50:29.790899+0000","last_became_active":"2023-06-08T03:50:24.691699+0000","last_became_peered":"2023-06-08T03:50:24.691699+0000","last_unstale":"2023-06-08T03:50:29.790899+0000","last_undegraded":"2023-06-08T03:50:29.790899+0000","last_fullsized":"2023-06-08T03:50:29.790899+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:40:30.017039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310163+0000","last_change":"2023-06-08T03:50:28.926277+0000","last_active":"2023-06-08T03:50:29.310163+0000","last_peered":"2023-06-08T03:50:29.310163+0000","last_clean":"2023-06-08T03:50:29.310163+0000","last_became_active":"2023-06-08T03:50:28.925921+0000","last_became_peered":"2023-06-08T03:50:28.925921+0000","last_unstale":"2023-06-08T03:50:29.310163+0000","last_undegraded":"2023-06-08T03:50:29.310163+0000","last_fullsized":"2023-06-08T03:50:29.310163+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:32:27.673747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1,6],"acting":[4,5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789845+0000","last_change":"2023-06-08T03:50:24.283127+0000","last_active":"2023-06-08T03:50:29.789845+0000","last_peered":"2023-06-08T03:50:29.789845+0000","last_clean":"2023-06-08T03:50:29.789845+0000","last_became_active":"2023-06-08T03:50:24.282515+0000","last_became_peered":"2023-06-08T03:50:24.282515+0000","last_unstale":"2023-06-08T03:50:29.789845+0000","last_undegraded":"2023-06-08T03:50:29.789845+0000","last_fullsized":"2023-06-08T03:50:29.789845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:55.345558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311940+0000","last_change":"2023-06-08T03:50:24.687906+0000","last_active":"2023-06-08T03:50:29.311940+0000","last_peered":"2023-06-08T03:50:29.311940+0000","last_clean":"2023-06-08T03:50:29.311940+0000","last_became_active":"2023-06-08T03:50:24.687630+0000","last_became_peered":"2023-06-08T03:50:24.687630+0000","last_unstale":"2023-06-08T03:50:29.311940+0000","last_undegraded":"2023-06-08T03:50:29.311940+0000","last_fullsized":"2023-06-08T03:50:29.311940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:12.100669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790030+0000","last_change":"2023-06-08T03:50:28.918543+0000","last_active":"2023-06-08T03:50:29.790030+0000","last_peered":"2023-06-08T03:50:29.790030+0000","last_clean":"2023-06-08T03:50:29.790030+0000","last_became_active":"2023-06-08T03:50:28.918124+0000","last_became_peered":"2023-06-08T03:50:28.918124+0000","last_unstale":"2023-06-08T03:50:29.790030+0000","last_undegraded":"2023-06-08T03:50:29.790030+0000","last_fullsized":"2023-06-08T03:50:29.790030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:32:30.430868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4,0],"acting":[3,6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789271+0000","last_change":"2023-06-08T03:50:24.286716+0000","last_active":"2023-06-08T03:50:29.789271+0000","last_peered":"2023-06-08T03:50:29.789271+0000","last_clean":"2023-06-08T03:50:29.789271+0000","last_became_active":"2023-06-08T03:50:24.286096+0000","last_became_peered":"2023-06-08T03:50:24.286096+0000","last_unstale":"2023-06-08T03:50:29.789271+0000","last_undegraded":"2023-06-08T03:50:29.789271+0000","last_fullsized":"2023-06-08T03:50:29.789271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:56:46.454730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312273+0000","last_change":"2023-06-08T03:50:24.690535+0000","last_active":"2023-06-08T03:50:29.312273+0000","last_peered":"2023-06-08T03:50:29.312273+0000","last_clean":"2023-06-08T03:50:29.312273+0000","last_became_active":"2023-06-08T03:50:24.689649+0000","last_became_peered":"2023-06-08T03:50:24.689649+0000","last_unstale":"2023-06-08T03:50:29.312273+0000","last_undegraded":"2023-06-08T03:50:29.312273+0000","last_fullsized":"2023-06-08T03:50:29.312273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:07.014455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789697+0000","last_change":"2023-06-08T03:50:28.926759+0000","last_active":"2023-06-08T03:50:29.789697+0000","last_peered":"2023-06-08T03:50:29.789697+0000","last_clean":"2023-06-08T03:50:29.789697+0000","last_became_active":"2023-06-08T03:50:28.926391+0000","last_became_peered":"2023-06-08T03:50:28.926391+0000","last_unstale":"2023-06-08T03:50:29.789697+0000","last_undegraded":"2023-06-08T03:50:29.789697+0000","last_fullsized":"2023-06-08T03:50:29.789697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:28:31.793415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,7],"acting":[1,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591134+0000","last_change":"2023-06-08T03:50:24.286445+0000","last_active":"2023-06-08T03:50:29.591134+0000","last_peered":"2023-06-08T03:50:29.591134+0000","last_clean":"2023-06-08T03:50:29.591134+0000","last_became_active":"2023-06-08T03:50:24.286062+0000","last_became_peered":"2023-06-08T03:50:24.286062+0000","last_unstale":"2023-06-08T03:50:29.591134+0000","last_undegraded":"2023-06-08T03:50:29.591134+0000","last_fullsized":"2023-06-08T03:50:29.591134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791098+0000","last_change":"2023-06-08T03:50:24.692233+0000","last_active":"2023-06-08T03:50:29.791098+0000","last_peered":"2023-06-08T03:50:29.791098+0000","last_clean":"2023-06-08T03:50:29.791098+0000","last_became_active":"2023-06-08T03:50:24.691584+0000","last_became_peered":"2023-06-08T03:50:24.691584+0000","last_unstale":"2023-06-08T03:50:29.791098+0000","last_undegraded":"2023-06-08T03:50:29.791098+0000","last_fullsized":"2023-06-08T03:50:29.791098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:57:43.387552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309967+0000","last_change":"2023-06-08T03:50:28.922207+0000","last_active":"2023-06-08T03:50:29.309967+0000","last_peered":"2023-06-08T03:50:29.309967+0000","last_clean":"2023-06-08T03:50:29.309967+0000","last_became_active":"2023-06-08T03:50:28.921809+0000","last_became_peered":"2023-06-08T03:50:28.921809+0000","last_unstale":"2023-06-08T03:50:29.309967+0000","last_undegraded":"2023-06-08T03:50:29.309967+0000","last_fullsized":"2023-06-08T03:50:29.309967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:23:47.506790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,0],"acting":[6,4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790800+0000","last_change":"2023-06-08T03:50:24.280168+0000","last_active":"2023-06-08T03:50:29.790800+0000","last_peered":"2023-06-08T03:50:29.790800+0000","last_clean":"2023-06-08T03:50:29.790800+0000","last_became_active":"2023-06-08T03:50:24.279947+0000","last_became_peered":"2023-06-08T03:50:24.279947+0000","last_unstale":"2023-06-08T03:50:29.790800+0000","last_undegraded":"2023-06-08T03:50:29.790800+0000","last_fullsized":"2023-06-08T03:50:29.790800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:32:11.091336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790420+0000","last_change":"2023-06-08T03:50:24.702952+0000","last_active":"2023-06-08T03:50:29.790420+0000","last_peered":"2023-06-08T03:50:29.790420+0000","last_clean":"2023-06-08T03:50:29.790420+0000","last_became_active":"2023-06-08T03:50:24.702213+0000","last_became_peered":"2023-06-08T03:50:24.702213+0000","last_unstale":"2023-06-08T03:50:29.790420+0000","last_undegraded":"2023-06-08T03:50:29.790420+0000","last_fullsized":"2023-06-08T03:50:29.790420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:57:59.540147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310770+0000","last_change":"2023-06-08T03:50:28.921768+0000","last_active":"2023-06-08T03:50:29.310770+0000","last_peered":"2023-06-08T03:50:29.310770+0000","last_clean":"2023-06-08T03:50:29.310770+0000","last_became_active":"2023-06-08T03:50:28.921477+0000","last_became_peered":"2023-06-08T03:50:28.921477+0000","last_unstale":"2023-06-08T03:50:29.310770+0000","last_undegraded":"2023-06-08T03:50:29.310770+0000","last_fullsized":"2023-06-08T03:50:29.310770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:06.300830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790697+0000","last_change":"2023-06-08T03:50:24.287775+0000","last_active":"2023-06-08T03:50:29.790697+0000","last_peered":"2023-06-08T03:50:29.790697+0000","last_clean":"2023-06-08T03:50:29.790697+0000","last_became_active":"2023-06-08T03:50:24.287136+0000","last_became_peered":"2023-06-08T03:50:24.287136+0000","last_unstale":"2023-06-08T03:50:29.790697+0000","last_undegraded":"2023-06-08T03:50:29.790697+0000","last_fullsized":"2023-06-08T03:50:29.790697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:17:17.207119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789501+0000","last_change":"2023-06-08T03:50:24.702234+0000","last_active":"2023-06-08T03:50:29.789501+0000","last_peered":"2023-06-08T03:50:29.789501+0000","last_clean":"2023-06-08T03:50:29.789501+0000","last_became_active":"2023-06-08T03:50:24.701881+0000","last_became_peered":"2023-06-08T03:50:24.701881+0000","last_unstale":"2023-06-08T03:50:29.789501+0000","last_undegraded":"2023-06-08T03:50:29.789501+0000","last_fullsized":"2023-06-08T03:50:29.789501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:52:37.709189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308237+0000","last_change":"2023-06-08T03:50:28.914578+0000","last_active":"2023-06-08T03:50:29.308237+0000","last_peered":"2023-06-08T03:50:29.308237+0000","last_clean":"2023-06-08T03:50:29.308237+0000","last_became_active":"2023-06-08T03:50:28.914064+0000","last_became_peered":"2023-06-08T03:50:28.914064+0000","last_unstale":"2023-06-08T03:50:29.308237+0000","last_undegraded":"2023-06-08T03:50:29.308237+0000","last_fullsized":"2023-06-08T03:50:29.308237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:52.212363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,7],"acting":[6,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590838+0000","last_change":"2023-06-08T03:50:24.283466+0000","last_active":"2023-06-08T03:50:29.590838+0000","last_peered":"2023-06-08T03:50:29.590838+0000","last_clean":"2023-06-08T03:50:29.590838+0000","last_became_active":"2023-06-08T03:50:24.283156+0000","last_became_peered":"2023-06-08T03:50:24.283156+0000","last_unstale":"2023-06-08T03:50:29.590838+0000","last_undegraded":"2023-06-08T03:50:29.590838+0000","last_fullsized":"2023-06-08T03:50:29.590838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:21:25.016985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310840+0000","last_change":"2023-06-08T03:50:24.680665+0000","last_active":"2023-06-08T03:50:29.310840+0000","last_peered":"2023-06-08T03:50:29.310840+0000","last_clean":"2023-06-08T03:50:29.310840+0000","last_became_active":"2023-06-08T03:50:24.680424+0000","last_became_peered":"2023-06-08T03:50:24.680424+0000","last_unstale":"2023-06-08T03:50:29.310840+0000","last_undegraded":"2023-06-08T03:50:29.310840+0000","last_fullsized":"2023-06-08T03:50:29.310840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:26:24.624745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789596+0000","last_change":"2023-06-08T03:50:28.919899+0000","last_active":"2023-06-08T03:50:29.789596+0000","last_peered":"2023-06-08T03:50:29.789596+0000","last_clean":"2023-06-08T03:50:29.789596+0000","last_became_active":"2023-06-08T03:50:28.919515+0000","last_became_peered":"2023-06-08T03:50:28.919515+0000","last_unstale":"2023-06-08T03:50:29.789596+0000","last_undegraded":"2023-06-08T03:50:29.789596+0000","last_fullsized":"2023-06-08T03:50:29.789596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:49:21.771214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5,1],"acting":[7,0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789597+0000","last_change":"2023-06-08T03:50:24.282752+0000","last_active":"2023-06-08T03:50:29.789597+0000","last_peered":"2023-06-08T03:50:29.789597+0000","last_clean":"2023-06-08T03:50:29.789597+0000","last_became_active":"2023-06-08T03:50:24.282473+0000","last_became_peered":"2023-06-08T03:50:24.282473+0000","last_unstale":"2023-06-08T03:50:29.789597+0000","last_undegraded":"2023-06-08T03:50:29.789597+0000","last_fullsized":"2023-06-08T03:50:29.789597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:15:57.351189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591060+0000","last_change":"2023-06-08T03:50:24.691781+0000","last_active":"2023-06-08T03:50:29.591060+0000","last_peered":"2023-06-08T03:50:29.591060+0000","last_clean":"2023-06-08T03:50:29.591060+0000","last_became_active":"2023-06-08T03:50:24.690055+0000","last_became_peered":"2023-06-08T03:50:24.690055+0000","last_unstale":"2023-06-08T03:50:29.591060+0000","last_undegraded":"2023-06-08T03:50:29.591060+0000","last_fullsized":"2023-06-08T03:50:29.591060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311854+0000","last_change":"2023-06-08T03:50:28.927198+0000","last_active":"2023-06-08T03:50:29.311854+0000","last_peered":"2023-06-08T03:50:29.311854+0000","last_clean":"2023-06-08T03:50:29.311854+0000","last_became_active":"2023-06-08T03:50:28.926812+0000","last_became_peered":"2023-06-08T03:50:28.926812+0000","last_unstale":"2023-06-08T03:50:29.311854+0000","last_undegraded":"2023-06-08T03:50:29.311854+0000","last_fullsized":"2023-06-08T03:50:29.311854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:20:44.857655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4,2],"acting":[5,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789506+0000","last_change":"2023-06-08T03:50:24.287767+0000","last_active":"2023-06-08T03:50:29.789506+0000","last_peered":"2023-06-08T03:50:29.789506+0000","last_clean":"2023-06-08T03:50:29.789506+0000","last_became_active":"2023-06-08T03:50:24.287060+0000","last_became_peered":"2023-06-08T03:50:24.287060+0000","last_unstale":"2023-06-08T03:50:29.789506+0000","last_undegraded":"2023-06-08T03:50:29.789506+0000","last_fullsized":"2023-06-08T03:50:29.789506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:15:27.592926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790578+0000","last_change":"2023-06-08T03:50:24.700545+0000","last_active":"2023-06-08T03:50:29.790578+0000","last_peered":"2023-06-08T03:50:29.790578+0000","last_clean":"2023-06-08T03:50:29.790578+0000","last_became_active":"2023-06-08T03:50:24.700276+0000","last_became_peered":"2023-06-08T03:50:24.700276+0000","last_unstale":"2023-06-08T03:50:29.790578+0000","last_undegraded":"2023-06-08T03:50:29.790578+0000","last_fullsized":"2023-06-08T03:50:29.790578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:55:50.252975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791082+0000","last_change":"2023-06-08T03:50:28.920817+0000","last_active":"2023-06-08T03:50:29.791082+0000","last_peered":"2023-06-08T03:50:29.791082+0000","last_clean":"2023-06-08T03:50:29.791082+0000","last_became_active":"2023-06-08T03:50:28.920485+0000","last_became_peered":"2023-06-08T03:50:28.920485+0000","last_unstale":"2023-06-08T03:50:29.791082+0000","last_undegraded":"2023-06-08T03:50:29.791082+0000","last_fullsized":"2023-06-08T03:50:29.791082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:36.826227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2,5],"acting":[1,0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790009+0000","last_change":"2023-06-08T03:50:24.283331+0000","last_active":"2023-06-08T03:50:29.790009+0000","last_peered":"2023-06-08T03:50:29.790009+0000","last_clean":"2023-06-08T03:50:29.790009+0000","last_became_active":"2023-06-08T03:50:24.282856+0000","last_became_peered":"2023-06-08T03:50:24.282856+0000","last_unstale":"2023-06-08T03:50:29.790009+0000","last_undegraded":"2023-06-08T03:50:29.790009+0000","last_fullsized":"2023-06-08T03:50:29.790009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:52:45.337874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790917+0000","last_change":"2023-06-08T03:50:24.683795+0000","last_active":"2023-06-08T03:50:29.790917+0000","last_peered":"2023-06-08T03:50:29.790917+0000","last_clean":"2023-06-08T03:50:29.790917+0000","last_became_active":"2023-06-08T03:50:24.683510+0000","last_became_peered":"2023-06-08T03:50:24.683510+0000","last_unstale":"2023-06-08T03:50:29.790917+0000","last_undegraded":"2023-06-08T03:50:29.790917+0000","last_fullsized":"2023-06-08T03:50:29.790917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790162+0000","last_change":"2023-06-08T03:50:28.920202+0000","last_active":"2023-06-08T03:50:29.790162+0000","last_peered":"2023-06-08T03:50:29.790162+0000","last_clean":"2023-06-08T03:50:29.790162+0000","last_became_active":"2023-06-08T03:50:28.919960+0000","last_became_peered":"2023-06-08T03:50:28.919960+0000","last_unstale":"2023-06-08T03:50:29.790162+0000","last_undegraded":"2023-06-08T03:50:29.790162+0000","last_fullsized":"2023-06-08T03:50:29.790162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:33:44.875605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4,5],"acting":[3,2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307873+0000","last_change":"2023-06-08T03:50:24.284333+0000","last_active":"2023-06-08T03:50:29.307873+0000","last_peered":"2023-06-08T03:50:29.307873+0000","last_clean":"2023-06-08T03:50:29.307873+0000","last_became_active":"2023-06-08T03:50:24.284039+0000","last_became_peered":"2023-06-08T03:50:24.284039+0000","last_unstale":"2023-06-08T03:50:29.307873+0000","last_undegraded":"2023-06-08T03:50:29.307873+0000","last_fullsized":"2023-06-08T03:50:29.307873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:37:13.194212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789927+0000","last_change":"2023-06-08T03:50:24.691444+0000","last_active":"2023-06-08T03:50:29.789927+0000","last_peered":"2023-06-08T03:50:29.789927+0000","last_clean":"2023-06-08T03:50:29.789927+0000","last_became_active":"2023-06-08T03:50:24.690971+0000","last_became_peered":"2023-06-08T03:50:24.690971+0000","last_unstale":"2023-06-08T03:50:29.789927+0000","last_undegraded":"2023-06-08T03:50:29.789927+0000","last_fullsized":"2023-06-08T03:50:29.789927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:38:23.297572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790270+0000","last_change":"2023-06-08T03:50:28.929890+0000","last_active":"2023-06-08T03:50:29.790270+0000","last_peered":"2023-06-08T03:50:29.790270+0000","last_clean":"2023-06-08T03:50:29.790270+0000","last_became_active":"2023-06-08T03:50:28.929496+0000","last_became_peered":"2023-06-08T03:50:28.929496+0000","last_unstale":"2023-06-08T03:50:29.790270+0000","last_undegraded":"2023-06-08T03:50:29.790270+0000","last_fullsized":"2023-06-08T03:50:29.790270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:21:51.680343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310379+0000","last_change":"2023-06-08T03:50:24.282724+0000","last_active":"2023-06-08T03:50:29.310379+0000","last_peered":"2023-06-08T03:50:29.310379+0000","last_clean":"2023-06-08T03:50:29.310379+0000","last_became_active":"2023-06-08T03:50:24.282451+0000","last_became_peered":"2023-06-08T03:50:24.282451+0000","last_unstale":"2023-06-08T03:50:29.310379+0000","last_undegraded":"2023-06-08T03:50:29.310379+0000","last_fullsized":"2023-06-08T03:50:29.310379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:44:39.775596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790625+0000","last_change":"2023-06-08T03:50:19.804887+0000","last_active":"2023-06-08T03:50:29.790625+0000","last_peered":"2023-06-08T03:50:29.790625+0000","last_clean":"2023-06-08T03:50:29.790625+0000","last_became_active":"2023-06-08T03:50:19.804546+0000","last_became_peered":"2023-06-08T03:50:19.804546+0000","last_unstale":"2023-06-08T03:50:29.790625+0000","last_undegraded":"2023-06-08T03:50:29.790625+0000","last_fullsized":"2023-06-08T03:50:29.790625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:34.859632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307694+0000","last_change":"2023-06-08T03:50:24.686843+0000","last_active":"2023-06-08T03:50:29.307694+0000","last_peered":"2023-06-08T03:50:29.307694+0000","last_clean":"2023-06-08T03:50:29.307694+0000","last_became_active":"2023-06-08T03:50:24.686587+0000","last_became_peered":"2023-06-08T03:50:24.686587+0000","last_unstale":"2023-06-08T03:50:29.307694+0000","last_undegraded":"2023-06-08T03:50:29.307694+0000","last_fullsized":"2023-06-08T03:50:29.307694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:36.573402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308557+0000","last_change":"2023-06-08T03:50:28.924976+0000","last_active":"2023-06-08T03:50:29.308557+0000","last_peered":"2023-06-08T03:50:29.308557+0000","last_clean":"2023-06-08T03:50:29.308557+0000","last_became_active":"2023-06-08T03:50:28.924335+0000","last_became_peered":"2023-06-08T03:50:28.924335+0000","last_unstale":"2023-06-08T03:50:29.308557+0000","last_undegraded":"2023-06-08T03:50:29.308557+0000","last_fullsized":"2023-06-08T03:50:29.308557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:47:44.099384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789646+0000","last_change":"2023-06-08T03:50:19.172271+0000","last_active":"2023-06-08T03:50:29.789646+0000","last_peered":"2023-06-08T03:50:29.789646+0000","last_clean":"2023-06-08T03:50:29.789646+0000","last_became_active":"2023-06-08T03:50:19.171917+0000","last_became_peered":"2023-06-08T03:50:19.171917+0000","last_unstale":"2023-06-08T03:50:29.789646+0000","last_undegraded":"2023-06-08T03:50:29.789646+0000","last_fullsized":"2023-06-08T03:50:29.789646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:01:56.709362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308387+0000","last_change":"2023-06-08T03:50:24.284253+0000","last_active":"2023-06-08T03:50:29.308387+0000","last_peered":"2023-06-08T03:50:29.308387+0000","last_clean":"2023-06-08T03:50:29.308387+0000","last_became_active":"2023-06-08T03:50:24.283958+0000","last_became_peered":"2023-06-08T03:50:24.283958+0000","last_unstale":"2023-06-08T03:50:29.308387+0000","last_undegraded":"2023-06-08T03:50:29.308387+0000","last_fullsized":"2023-06-08T03:50:29.308387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:02:34.479410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.690498+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.690242+0000","last_became_peered":"2023-06-08T03:50:24.690242+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:23:08.690780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790091+0000","last_change":"2023-06-08T03:50:28.921864+0000","last_active":"2023-06-08T03:50:29.790091+0000","last_peered":"2023-06-08T03:50:29.790091+0000","last_clean":"2023-06-08T03:50:29.790091+0000","last_became_active":"2023-06-08T03:50:28.921426+0000","last_became_peered":"2023-06-08T03:50:28.921426+0000","last_unstale":"2023-06-08T03:50:29.790091+0000","last_undegraded":"2023-06-08T03:50:29.790091+0000","last_fullsized":"2023-06-08T03:50:29.790091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:43:24.342082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,0],"acting":[3,7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309435+0000","last_change":"2023-06-08T03:50:19.171073+0000","last_active":"2023-06-08T03:50:29.309435+0000","last_peered":"2023-06-08T03:50:29.309435+0000","last_clean":"2023-06-08T03:50:29.309435+0000","last_became_active":"2023-06-08T03:50:19.170775+0000","last_became_peered":"2023-06-08T03:50:19.170775+0000","last_unstale":"2023-06-08T03:50:29.309435+0000","last_undegraded":"2023-06-08T03:50:29.309435+0000","last_fullsized":"2023-06-08T03:50:29.309435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:51:24.758705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.791212+0000","last_change":"2023-06-08T03:50:19.167800+0000","last_active":"2023-06-08T03:50:29.791212+0000","last_peered":"2023-06-08T03:50:29.791212+0000","last_clean":"2023-06-08T03:50:29.791212+0000","last_became_active":"2023-06-08T03:50:19.167412+0000","last_became_peered":"2023-06-08T03:50:19.167412+0000","last_unstale":"2023-06-08T03:50:29.791212+0000","last_undegraded":"2023-06-08T03:50:29.791212+0000","last_fullsized":"2023-06-08T03:50:29.791212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:45:23.552252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309769+0000","last_change":"2023-06-08T03:50:24.280635+0000","last_active":"2023-06-08T03:50:29.309769+0000","last_peered":"2023-06-08T03:50:29.309769+0000","last_clean":"2023-06-08T03:50:29.309769+0000","last_became_active":"2023-06-08T03:50:24.280376+0000","last_became_peered":"2023-06-08T03:50:24.280376+0000","last_unstale":"2023-06-08T03:50:29.309769+0000","last_undegraded":"2023-06-08T03:50:29.309769+0000","last_fullsized":"2023-06-08T03:50:29.309769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:41.269585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312281+0000","last_change":"2023-06-08T03:50:24.685266+0000","last_active":"2023-06-08T03:50:29.312281+0000","last_peered":"2023-06-08T03:50:29.312281+0000","last_clean":"2023-06-08T03:50:29.312281+0000","last_became_active":"2023-06-08T03:50:24.684970+0000","last_became_peered":"2023-06-08T03:50:24.684970+0000","last_unstale":"2023-06-08T03:50:29.312281+0000","last_undegraded":"2023-06-08T03:50:29.312281+0000","last_fullsized":"2023-06-08T03:50:29.312281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789896+0000","last_change":"2023-06-08T03:50:28.926537+0000","last_active":"2023-06-08T03:50:29.789896+0000","last_peered":"2023-06-08T03:50:29.789896+0000","last_clean":"2023-06-08T03:50:29.789896+0000","last_became_active":"2023-06-08T03:50:28.926095+0000","last_became_peered":"2023-06-08T03:50:28.926095+0000","last_unstale":"2023-06-08T03:50:29.789896+0000","last_undegraded":"2023-06-08T03:50:29.789896+0000","last_fullsized":"2023-06-08T03:50:29.789896+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:16:59.025169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,1],"acting":[7,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.590877+0000","last_change":"2023-06-08T03:50:19.171834+0000","last_active":"2023-06-08T03:50:29.590877+0000","last_peered":"2023-06-08T03:50:29.590877+0000","last_clean":"2023-06-08T03:50:29.590877+0000","last_became_active":"2023-06-08T03:50:19.171541+0000","last_became_peered":"2023-06-08T03:50:19.171541+0000","last_unstale":"2023-06-08T03:50:29.590877+0000","last_undegraded":"2023-06-08T03:50:29.590877+0000","last_fullsized":"2023-06-08T03:50:29.590877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:42.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310150+0000","last_change":"2023-06-08T03:50:24.275533+0000","last_active":"2023-06-08T03:50:29.310150+0000","last_peered":"2023-06-08T03:50:29.310150+0000","last_clean":"2023-06-08T03:50:29.310150+0000","last_became_active":"2023-06-08T03:50:24.275086+0000","last_became_peered":"2023-06-08T03:50:24.275086+0000","last_unstale":"2023-06-08T03:50:29.310150+0000","last_undegraded":"2023-06-08T03:50:29.310150+0000","last_fullsized":"2023-06-08T03:50:29.310150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:49.514141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309093+0000","last_change":"2023-06-08T03:50:24.694692+0000","last_active":"2023-06-08T03:50:29.309093+0000","last_peered":"2023-06-08T03:50:29.309093+0000","last_clean":"2023-06-08T03:50:29.309093+0000","last_became_active":"2023-06-08T03:50:24.694404+0000","last_became_peered":"2023-06-08T03:50:24.694404+0000","last_unstale":"2023-06-08T03:50:29.309093+0000","last_undegraded":"2023-06-08T03:50:29.309093+0000","last_fullsized":"2023-06-08T03:50:29.309093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:06.873559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591398+0000","last_change":"2023-06-08T03:50:28.926999+0000","last_active":"2023-06-08T03:50:29.591398+0000","last_peered":"2023-06-08T03:50:29.591398+0000","last_clean":"2023-06-08T03:50:29.591398+0000","last_became_active":"2023-06-08T03:50:28.926614+0000","last_became_peered":"2023-06-08T03:50:28.926614+0000","last_unstale":"2023-06-08T03:50:29.591398+0000","last_undegraded":"2023-06-08T03:50:29.591398+0000","last_fullsized":"2023-06-08T03:50:29.591398+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:34:45.910533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789240+0000","last_change":"2023-06-08T03:50:19.170601+0000","last_active":"2023-06-08T03:50:29.789240+0000","last_peered":"2023-06-08T03:50:29.789240+0000","last_clean":"2023-06-08T03:50:29.789240+0000","last_became_active":"2023-06-08T03:50:19.170344+0000","last_became_peered":"2023-06-08T03:50:19.170344+0000","last_unstale":"2023-06-08T03:50:29.789240+0000","last_undegraded":"2023-06-08T03:50:29.789240+0000","last_fullsized":"2023-06-08T03:50:29.789240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:35.924638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.307871+0000","last_change":"2023-06-08T03:50:24.280451+0000","last_active":"2023-06-08T03:50:29.307871+0000","last_peered":"2023-06-08T03:50:29.307871+0000","last_clean":"2023-06-08T03:50:29.307871+0000","last_became_active":"2023-06-08T03:50:24.280118+0000","last_became_peered":"2023-06-08T03:50:24.280118+0000","last_unstale":"2023-06-08T03:50:29.307871+0000","last_undegraded":"2023-06-08T03:50:29.307871+0000","last_fullsized":"2023-06-08T03:50:29.307871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:18.146699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790813+0000","last_change":"2023-06-08T03:50:24.692660+0000","last_active":"2023-06-08T03:50:29.790813+0000","last_peered":"2023-06-08T03:50:29.790813+0000","last_clean":"2023-06-08T03:50:29.790813+0000","last_became_active":"2023-06-08T03:50:24.692373+0000","last_became_peered":"2023-06-08T03:50:24.692373+0000","last_unstale":"2023-06-08T03:50:29.790813+0000","last_undegraded":"2023-06-08T03:50:29.790813+0000","last_fullsized":"2023-06-08T03:50:29.790813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:14:48.949961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789804+0000","last_change":"2023-06-08T03:50:28.924634+0000","last_active":"2023-06-08T03:50:29.789804+0000","last_peered":"2023-06-08T03:50:29.789804+0000","last_clean":"2023-06-08T03:50:29.789804+0000","last_became_active":"2023-06-08T03:50:28.924227+0000","last_became_peered":"2023-06-08T03:50:28.924227+0000","last_unstale":"2023-06-08T03:50:29.789804+0000","last_undegraded":"2023-06-08T03:50:29.789804+0000","last_fullsized":"2023-06-08T03:50:29.789804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:24:05.389046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3,1],"acting":[7,4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789742+0000","last_change":"2023-06-08T03:50:19.805195+0000","last_active":"2023-06-08T03:50:29.789742+0000","last_peered":"2023-06-08T03:50:29.789742+0000","last_clean":"2023-06-08T03:50:29.789742+0000","last_became_active":"2023-06-08T03:50:19.804708+0000","last_became_peered":"2023-06-08T03:50:19.804708+0000","last_unstale":"2023-06-08T03:50:29.789742+0000","last_undegraded":"2023-06-08T03:50:29.789742+0000","last_fullsized":"2023-06-08T03:50:29.789742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:53:26.398016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591012+0000","last_change":"2023-06-08T03:50:24.281698+0000","last_active":"2023-06-08T03:50:29.591012+0000","last_peered":"2023-06-08T03:50:29.591012+0000","last_clean":"2023-06-08T03:50:29.591012+0000","last_became_active":"2023-06-08T03:50:24.281443+0000","last_became_peered":"2023-06-08T03:50:24.281443+0000","last_unstale":"2023-06-08T03:50:29.591012+0000","last_undegraded":"2023-06-08T03:50:29.591012+0000","last_fullsized":"2023-06-08T03:50:29.591012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:17:29.437443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309408+0000","last_change":"2023-06-08T03:50:24.693777+0000","last_active":"2023-06-08T03:50:29.309408+0000","last_peered":"2023-06-08T03:50:29.309408+0000","last_clean":"2023-06-08T03:50:29.309408+0000","last_became_active":"2023-06-08T03:50:24.693193+0000","last_became_peered":"2023-06-08T03:50:24.693193+0000","last_unstale":"2023-06-08T03:50:29.309408+0000","last_undegraded":"2023-06-08T03:50:29.309408+0000","last_fullsized":"2023-06-08T03:50:29.309408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:32:02.522778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591049+0000","last_change":"2023-06-08T03:50:28.926459+0000","last_active":"2023-06-08T03:50:29.591049+0000","last_peered":"2023-06-08T03:50:29.591049+0000","last_clean":"2023-06-08T03:50:29.591049+0000","last_became_active":"2023-06-08T03:50:28.926041+0000","last_became_peered":"2023-06-08T03:50:28.926041+0000","last_unstale":"2023-06-08T03:50:29.591049+0000","last_undegraded":"2023-06-08T03:50:29.591049+0000","last_fullsized":"2023-06-08T03:50:29.591049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:52:10.505175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2,5],"acting":[0,7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790298+0000","last_change":"2023-06-08T03:50:19.805012+0000","last_active":"2023-06-08T03:50:29.790298+0000","last_peered":"2023-06-08T03:50:29.790298+0000","last_clean":"2023-06-08T03:50:29.790298+0000","last_became_active":"2023-06-08T03:50:19.804722+0000","last_became_peered":"2023-06-08T03:50:19.804722+0000","last_unstale":"2023-06-08T03:50:29.790298+0000","last_undegraded":"2023-06-08T03:50:29.790298+0000","last_fullsized":"2023-06-08T03:50:29.790298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:43:32.916770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309261+0000","last_change":"2023-06-08T03:50:24.284390+0000","last_active":"2023-06-08T03:50:29.309261+0000","last_peered":"2023-06-08T03:50:29.309261+0000","last_clean":"2023-06-08T03:50:29.309261+0000","last_became_active":"2023-06-08T03:50:24.284129+0000","last_became_peered":"2023-06-08T03:50:24.284129+0000","last_unstale":"2023-06-08T03:50:29.309261+0000","last_undegraded":"2023-06-08T03:50:29.309261+0000","last_fullsized":"2023-06-08T03:50:29.309261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:12:03.192860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309328+0000","last_change":"2023-06-08T03:50:24.693789+0000","last_active":"2023-06-08T03:50:29.309328+0000","last_peered":"2023-06-08T03:50:29.309328+0000","last_clean":"2023-06-08T03:50:29.309328+0000","last_became_active":"2023-06-08T03:50:24.693205+0000","last_became_peered":"2023-06-08T03:50:24.693205+0000","last_unstale":"2023-06-08T03:50:29.309328+0000","last_undegraded":"2023-06-08T03:50:29.309328+0000","last_fullsized":"2023-06-08T03:50:29.309328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:45:07.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310499+0000","last_change":"2023-06-08T03:50:28.924218+0000","last_active":"2023-06-08T03:50:29.310499+0000","last_peered":"2023-06-08T03:50:29.310499+0000","last_clean":"2023-06-08T03:50:29.310499+0000","last_became_active":"2023-06-08T03:50:28.923473+0000","last_became_peered":"2023-06-08T03:50:28.923473+0000","last_unstale":"2023-06-08T03:50:29.310499+0000","last_undegraded":"2023-06-08T03:50:29.310499+0000","last_fullsized":"2023-06-08T03:50:29.310499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:11:10.428932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3,1],"acting":[4,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790922+0000","last_change":"2023-06-08T03:50:19.805107+0000","last_active":"2023-06-08T03:50:29.790922+0000","last_peered":"2023-06-08T03:50:29.790922+0000","last_clean":"2023-06-08T03:50:29.790922+0000","last_became_active":"2023-06-08T03:50:19.804754+0000","last_became_peered":"2023-06-08T03:50:19.804754+0000","last_unstale":"2023-06-08T03:50:29.790922+0000","last_undegraded":"2023-06-08T03:50:29.790922+0000","last_fullsized":"2023-06-08T03:50:29.790922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:13.049279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790682+0000","last_change":"2023-06-08T03:50:24.281311+0000","last_active":"2023-06-08T03:50:29.790682+0000","last_peered":"2023-06-08T03:50:29.790682+0000","last_clean":"2023-06-08T03:50:29.790682+0000","last_became_active":"2023-06-08T03:50:24.280858+0000","last_became_peered":"2023-06-08T03:50:24.280858+0000","last_unstale":"2023-06-08T03:50:29.790682+0000","last_undegraded":"2023-06-08T03:50:29.790682+0000","last_fullsized":"2023-06-08T03:50:29.790682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:01.113573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309301+0000","last_change":"2023-06-08T03:50:24.700010+0000","last_active":"2023-06-08T03:50:29.309301+0000","last_peered":"2023-06-08T03:50:29.309301+0000","last_clean":"2023-06-08T03:50:29.309301+0000","last_became_active":"2023-06-08T03:50:24.699801+0000","last_became_peered":"2023-06-08T03:50:24.699801+0000","last_unstale":"2023-06-08T03:50:29.309301+0000","last_undegraded":"2023-06-08T03:50:29.309301+0000","last_fullsized":"2023-06-08T03:50:29.309301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:48.012586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308391+0000","last_change":"2023-06-08T03:50:28.920073+0000","last_active":"2023-06-08T03:50:29.308391+0000","last_peered":"2023-06-08T03:50:29.308391+0000","last_clean":"2023-06-08T03:50:29.308391+0000","last_became_active":"2023-06-08T03:50:28.919601+0000","last_became_peered":"2023-06-08T03:50:28.919601+0000","last_unstale":"2023-06-08T03:50:29.308391+0000","last_undegraded":"2023-06-08T03:50:29.308391+0000","last_fullsized":"2023-06-08T03:50:29.308391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:38:02.965137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309419+0000","last_change":"2023-06-08T03:50:24.285734+0000","last_active":"2023-06-08T03:50:29.309419+0000","last_peered":"2023-06-08T03:50:29.309419+0000","last_clean":"2023-06-08T03:50:29.309419+0000","last_became_active":"2023-06-08T03:50:24.285421+0000","last_became_peered":"2023-06-08T03:50:24.285421+0000","last_unstale":"2023-06-08T03:50:29.309419+0000","last_undegraded":"2023-06-08T03:50:29.309419+0000","last_fullsized":"2023-06-08T03:50:29.309419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:13:35.050524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309586+0000","last_change":"2023-06-08T03:50:24.696432+0000","last_active":"2023-06-08T03:50:29.309586+0000","last_peered":"2023-06-08T03:50:29.309586+0000","last_clean":"2023-06-08T03:50:29.309586+0000","last_became_active":"2023-06-08T03:50:24.695650+0000","last_became_peered":"2023-06-08T03:50:24.695650+0000","last_unstale":"2023-06-08T03:50:29.309586+0000","last_undegraded":"2023-06-08T03:50:29.309586+0000","last_fullsized":"2023-06-08T03:50:29.309586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:33:18.696861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790496+0000","last_change":"2023-06-08T03:50:28.929940+0000","last_active":"2023-06-08T03:50:29.790496+0000","last_peered":"2023-06-08T03:50:29.790496+0000","last_clean":"2023-06-08T03:50:29.790496+0000","last_became_active":"2023-06-08T03:50:28.929641+0000","last_became_peered":"2023-06-08T03:50:28.929641+0000","last_unstale":"2023-06-08T03:50:29.790496+0000","last_undegraded":"2023-06-08T03:50:29.790496+0000","last_fullsized":"2023-06-08T03:50:29.790496+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:13:40.205018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789194+0000","last_change":"2023-06-08T03:50:24.285199+0000","last_active":"2023-06-08T03:50:29.789194+0000","last_peered":"2023-06-08T03:50:29.789194+0000","last_clean":"2023-06-08T03:50:29.789194+0000","last_became_active":"2023-06-08T03:50:24.284917+0000","last_became_peered":"2023-06-08T03:50:24.284917+0000","last_unstale":"2023-06-08T03:50:29.789194+0000","last_undegraded":"2023-06-08T03:50:29.789194+0000","last_fullsized":"2023-06-08T03:50:29.789194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790180+0000","last_change":"2023-06-08T03:50:24.680708+0000","last_active":"2023-06-08T03:50:29.790180+0000","last_peered":"2023-06-08T03:50:29.790180+0000","last_clean":"2023-06-08T03:50:29.790180+0000","last_became_active":"2023-06-08T03:50:24.680437+0000","last_became_peered":"2023-06-08T03:50:24.680437+0000","last_unstale":"2023-06-08T03:50:29.790180+0000","last_undegraded":"2023-06-08T03:50:29.790180+0000","last_fullsized":"2023-06-08T03:50:29.790180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:10:44.278509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789986+0000","last_change":"2023-06-08T03:50:28.922303+0000","last_active":"2023-06-08T03:50:29.789986+0000","last_peered":"2023-06-08T03:50:29.789986+0000","last_clean":"2023-06-08T03:50:29.789986+0000","last_became_active":"2023-06-08T03:50:28.921661+0000","last_became_peered":"2023-06-08T03:50:28.921661+0000","last_unstale":"2023-06-08T03:50:29.789986+0000","last_undegraded":"2023-06-08T03:50:29.789986+0000","last_fullsized":"2023-06-08T03:50:29.789986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:00:54.155089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3,4],"acting":[2,7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.790594+0000","last_change":"2023-06-08T03:50:24.287963+0000","last_active":"2023-06-08T03:50:29.790594+0000","last_peered":"2023-06-08T03:50:29.790594+0000","last_clean":"2023-06-08T03:50:29.790594+0000","last_became_active":"2023-06-08T03:50:24.287513+0000","last_became_peered":"2023-06-08T03:50:24.287513+0000","last_unstale":"2023-06-08T03:50:29.790594+0000","last_undegraded":"2023-06-08T03:50:29.790594+0000","last_fullsized":"2023-06-08T03:50:29.790594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:41:31.366674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789388+0000","last_change":"2023-06-08T03:50:24.694656+0000","last_active":"2023-06-08T03:50:29.789388+0000","last_peered":"2023-06-08T03:50:29.789388+0000","last_clean":"2023-06-08T03:50:29.789388+0000","last_became_active":"2023-06-08T03:50:24.694387+0000","last_became_peered":"2023-06-08T03:50:24.694387+0000","last_unstale":"2023-06-08T03:50:29.789388+0000","last_undegraded":"2023-06-08T03:50:29.789388+0000","last_fullsized":"2023-06-08T03:50:29.789388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:11:50.398160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789148+0000","last_change":"2023-06-08T03:50:28.926194+0000","last_active":"2023-06-08T03:50:29.789148+0000","last_peered":"2023-06-08T03:50:29.789148+0000","last_clean":"2023-06-08T03:50:29.789148+0000","last_became_active":"2023-06-08T03:50:28.925783+0000","last_became_peered":"2023-06-08T03:50:28.925783+0000","last_unstale":"2023-06-08T03:50:29.789148+0000","last_undegraded":"2023-06-08T03:50:29.789148+0000","last_fullsized":"2023-06-08T03:50:29.789148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:26.774200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5,7],"acting":[1,0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312099+0000","last_change":"2023-06-08T03:50:24.277779+0000","last_active":"2023-06-08T03:50:29.312099+0000","last_peered":"2023-06-08T03:50:29.312099+0000","last_clean":"2023-06-08T03:50:29.312099+0000","last_became_active":"2023-06-08T03:50:24.277493+0000","last_became_peered":"2023-06-08T03:50:24.277493+0000","last_unstale":"2023-06-08T03:50:29.312099+0000","last_undegraded":"2023-06-08T03:50:29.312099+0000","last_fullsized":"2023-06-08T03:50:29.312099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:27:23.390833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789292+0000","last_change":"2023-06-08T03:50:24.696150+0000","last_active":"2023-06-08T03:50:29.789292+0000","last_peered":"2023-06-08T03:50:29.789292+0000","last_clean":"2023-06-08T03:50:29.789292+0000","last_became_active":"2023-06-08T03:50:24.695605+0000","last_became_peered":"2023-06-08T03:50:24.695605+0000","last_unstale":"2023-06-08T03:50:29.789292+0000","last_undegraded":"2023-06-08T03:50:29.789292+0000","last_fullsized":"2023-06-08T03:50:29.789292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:28:06.091963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.308122+0000","last_change":"2023-06-08T03:50:28.924072+0000","last_active":"2023-06-08T03:50:29.308122+0000","last_peered":"2023-06-08T03:50:29.308122+0000","last_clean":"2023-06-08T03:50:29.308122+0000","last_became_active":"2023-06-08T03:50:28.923155+0000","last_became_peered":"2023-06-08T03:50:28.923155+0000","last_unstale":"2023-06-08T03:50:29.308122+0000","last_undegraded":"2023-06-08T03:50:29.308122+0000","last_fullsized":"2023-06-08T03:50:29.308122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:27.890885+0000","last_clean_scrub_stamp":"2023-06-08T03:50:27.890885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:00:25.670438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1,0],"acting":[4,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.312115+0000","last_change":"2023-06-08T03:50:24.285764+0000","last_active":"2023-06-08T03:50:29.312115+0000","last_peered":"2023-06-08T03:50:29.312115+0000","last_clean":"2023-06-08T03:50:29.312115+0000","last_became_active":"2023-06-08T03:50:24.285477+0000","last_became_peered":"2023-06-08T03:50:24.285477+0000","last_unstale":"2023-06-08T03:50:29.312115+0000","last_undegraded":"2023-06-08T03:50:29.312115+0000","last_fullsized":"2023-06-08T03:50:29.312115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:23.249937+0000","last_clean_scrub_stamp":"2023-06-08T03:50:23.249937+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:39:41.022058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.788785+0000","last_change":"2023-06-08T03:50:24.679506+0000","last_active":"2023-06-08T03:50:29.788785+0000","last_peered":"2023-06-08T03:50:29.788785+0000","last_clean":"2023-06-08T03:50:29.788785+0000","last_became_active":"2023-06-08T03:50:24.679246+0000","last_became_peered":"2023-06-08T03:50:24.679246+0000","last_unstale":"2023-06-08T03:50:29.788785+0000","last_undegraded":"2023-06-08T03:50:29.788785+0000","last_fullsized":"2023-06-08T03:50:29.788785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:53:20.880498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.789376+0000","last_change":"2023-06-08T03:50:24.690393+0000","last_active":"2023-06-08T03:50:29.789376+0000","last_peered":"2023-06-08T03:50:29.789376+0000","last_clean":"2023-06-08T03:50:29.789376+0000","last_became_active":"2023-06-08T03:50:24.690144+0000","last_became_peered":"2023-06-08T03:50:24.690144+0000","last_unstale":"2023-06-08T03:50:29.789376+0000","last_undegraded":"2023-06-08T03:50:29.789376+0000","last_fullsized":"2023-06-08T03:50:29.789376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:06:48.218896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.309500+0000","last_change":"2023-06-08T03:50:24.693278+0000","last_active":"2023-06-08T03:50:29.309500+0000","last_peered":"2023-06-08T03:50:29.309500+0000","last_clean":"2023-06-08T03:50:29.309500+0000","last_became_active":"2023-06-08T03:50:24.692978+0000","last_became_peered":"2023-06-08T03:50:24.692978+0000","last_unstale":"2023-06-08T03:50:29.309500+0000","last_undegraded":"2023-06-08T03:50:29.309500+0000","last_fullsized":"2023-06-08T03:50:29.309500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:51:02.355658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.591028+0000","last_change":"2023-06-08T03:50:24.700428+0000","last_active":"2023-06-08T03:50:29.591028+0000","last_peered":"2023-06-08T03:50:29.591028+0000","last_clean":"2023-06-08T03:50:29.591028+0000","last_became_active":"2023-06-08T03:50:24.700158+0000","last_became_peered":"2023-06-08T03:50:24.700158+0000","last_unstale":"2023-06-08T03:50:29.591028+0000","last_undegraded":"2023-06-08T03:50:29.591028+0000","last_fullsized":"2023-06-08T03:50:29.591028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:34.318549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.310861+0000","last_change":"2023-06-08T03:50:24.690477+0000","last_active":"2023-06-08T03:50:29.310861+0000","last_peered":"2023-06-08T03:50:29.310861+0000","last_clean":"2023-06-08T03:50:29.310861+0000","last_became_active":"2023-06-08T03:50:24.690125+0000","last_became_peered":"2023-06-08T03:50:24.690125+0000","last_unstale":"2023-06-08T03:50:29.310861+0000","last_undegraded":"2023-06-08T03:50:29.310861+0000","last_fullsized":"2023-06-08T03:50:29.310861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:05:40.966384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-06-08T03:50:29.311958+0000","last_change":"2023-06-08T03:50:24.682936+0000","last_active":"2023-06-08T03:50:29.311958+0000","last_peered":"2023-06-08T03:50:29.311958+0000","last_clean":"2023-06-08T03:50:29.311958+0000","last_became_active":"2023-06-08T03:50:24.681545+0000","last_became_peered":"2023-06-08T03:50:24.681545+0000","last_unstale":"2023-06-08T03:50:29.311958+0000","last_undegraded":"2023-06-08T03:50:29.311958+0000","last_fullsized":"2023-06-08T03:50:29.311958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:24.259269+0000","last_clean_scrub_stamp":"2023-06-08T03:50:24.259269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:08:40.264342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4592,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.444}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":270336,"data_stored":109499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":110083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0630000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1689999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1340000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":109555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":278528,"data_stored":113203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5149999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4210000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3360000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.2990000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":880640,"data_stored":703675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":266240,"data_stored":109409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":700359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-08T03:50:48.046 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-08T03:50:48.047 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-08T03:50:48.047 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-08T03:50:48.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-08T03:50:48.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.194+0000 7f6cbcd8e700 1 Processor -- start 2023-06-08T03:50:48.198 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.194+0000 7f6cbcd8e700 1 -- start start 2023-06-08T03:50:48.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cbcd8e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6cb8159270 0x7f6cb80cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cbcd8e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80b16f0 0x7f6cb815cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cbcd8e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb806ae60 con 0x7f6cb80ca630 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cbcd8e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb80cebc0 con 0x7f6cb8159270 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cbcd8e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb806b9e0 con 0x7f6cb80b16f0 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb659c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6cb8159270 0x7f6cb80cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb5d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6cb80ca630 0x7f6cb80b0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58820/0 (socket says 172.21.15.79:58820) 2023-06-08T03:50:48.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb5d9b700 1 -- 172.21.15.79:0/628121511 learned_addr learned my addr 172.21.15.79:0/628121511 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb6d9d700 1 --2- 172.21.15.79:0/628121511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80b16f0 0x7f6cb815cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb659c700 1 -- 172.21.15.79:0/628121511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80b16f0 msgr2=0x7f6cb815cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb659c700 1 --2- 172.21.15.79:0/628121511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80b16f0 0x7f6cb815cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb659c700 1 -- 172.21.15.79:0/628121511 >> v1:172.21.15.79:6789/0 conn(0x7f6cb80ca630 legacy=0x7f6cb80b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb659c700 1 -- 172.21.15.79:0/628121511 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb806b0b0 con 0x7f6cb8159270 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb559a700 1 -- 172.21.15.79:0/628121511 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358185113 0 0) 0x7f6cb806ae60 con 0x7f6cb80ca630 2023-06-08T03:50:48.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb659c700 1 --2- 172.21.15.79:0/628121511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb80cd460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009840 tx=0x7f6ca8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e2bfb24fa5d661 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb559a700 1 -- 172.21.15.79:0/628121511 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8011830 con 0x7f6cb8159270 2023-06-08T03:50:48.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.195+0000 7f6cb559a700 1 -- 172.21.15.79:0/628121511 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8011990 con 0x7f6cb8159270 2023-06-08T03:50:48.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cb559a700 1 -- 172.21.15.79:0/628121511 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8011d20 con 0x7f6cb8159270 2023-06-08T03:50:48.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/628121511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 msgr2=0x7f6cb80cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/628121511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb80cd460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009840 tx=0x7f6ca8009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:48.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/628121511 shutdown_connections 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/628121511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80b16f0 0x7f6cb815cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/628121511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb80cd460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/628121511 >> 172.21.15.79:0/628121511 conn(0x7f6cb80cbfe0 msgr2=0x7f6cb80cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/628121511 shutdown_connections 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.196+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/628121511 wait complete. 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.197+0000 7f6cbcd8e700 1 Processor -- start 2023-06-08T03:50:48.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 -- start start 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cb80b16f0 0x7f6cb81538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80ca630 0x7f6cb815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb81e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb8159640 con 0x7f6cb8159270 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb815a930 con 0x7f6cb80b16f0 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cbcd8e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb8159890 con 0x7f6cb80ca630 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cb6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb81e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cb6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb81e5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35352/0 (socket says 172.21.15.79:35352) 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cb6d9d700 1 -- 172.21.15.79:0/1876722303 learned_addr learned my addr 172.21.15.79:0/1876722303 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cb659c700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cb80b16f0 0x7f6cb81538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.198+0000 7f6cb5d9b700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80ca630 0x7f6cb815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cb6d9d700 1 -- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80ca630 msgr2=0x7f6cb815b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cb6d9d700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80ca630 0x7f6cb815b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cb6d9d700 1 -- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cb80b16f0 msgr2=0x7f6cb81538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cb6d9d700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cb80b16f0 0x7f6cb81538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cb6d9d700 1 -- 172.21.15.79:0/1876722303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80094a0 con 0x7f6cb8159270 2023-06-08T03:50:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cb6d9d700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb81e5ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007de0 tx=0x7f6cb000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb000ed30 con 0x7f6cb8159270 2023-06-08T03:50:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0014420 con 0x7f6cb8159270 2023-06-08T03:50:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb8159b70 con 0x7f6cb8159270 2023-06-08T03:50:48.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb00147d0 con 0x7f6cb8159270 2023-06-08T03:50:48.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.199+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb815a130 con 0x7f6cb8159270 2023-06-08T03:50:48.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.200+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb8073e90 con 0x7f6cb8159270 2023-06-08T03:50:48.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.200+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6cb0014930 con 0x7f6cb8159270 2023-06-08T03:50:48.209 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.201+0000 7f6ca6ffd700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c9806ace0 0x7f6c9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.201+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6cb0089dd0 con 0x7f6cb8159270 2023-06-08T03:50:48.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.201+0000 7f6cb659c700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c9806ace0 0x7f6c9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.201+0000 7f6cb659c700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c9806ace0 0x7f6c9806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80090b0 tx=0x7f6ca801c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.205+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6cb0054c20 con 0x7f6cb8159270 2023-06-08T03:50:48.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.433+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6cb80b54e0 con 0x7f6cb8159270 2023-06-08T03:50:48.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.433+0000 7f6ca6ffd700 1 -- 172.21.15.79:0/1876722303 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6cb0058240 con 0x7f6cb8159270 2023-06-08T03:50:48.438 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:48.438 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-08T03:50:48.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c9806ace0 msgr2=0x7f6c9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c9806ace0 0x7f6c9806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80090b0 tx=0x7f6ca801c040 comp rx=0 tx=0).stop 2023-06-08T03:50:48.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 msgr2=0x7f6cb81e5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb81e5ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0007de0 tx=0x7f6cb000e3c0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 shutdown_connections 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cb8159270 0x7f6cb81e5ac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6cb80b16f0 0x7f6cb81538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c9806ace0 0x7f6c9806d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 --2- 172.21.15.79:0/1876722303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6cb80ca630 0x7f6cb815b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 >> 172.21.15.79:0/1876722303 conn(0x7f6cb80cbfe0 msgr2=0x7f6cb80caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 shutdown_connections 2023-06-08T03:50:48.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.436+0000 7f6cbcd8e700 1 -- 172.21.15.79:0/1876722303 wait complete. 2023-06-08T03:50:48.451 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-08T03:50:48.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:48.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.568+0000 7f627ab1b700 1 Processor -- start 2023-06-08T03:50:48.572 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.568+0000 7f627ab1b700 1 -- start start 2023-06-08T03:50:48.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f627ab1b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f627415a640 0x7f62740b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f627ab1b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f62740ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f627ab1b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62740b0750 con 0x7f62740ce930 2023-06-08T03:50:48.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f627ab1b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62740ceeb0 con 0x7f627415a640 2023-06-08T03:50:48.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f627ab1b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62740b09a0 con 0x7f627415aa10 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f627415a640 0x7f62740b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f627415a640 0x7f62740b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35360/0 (socket says 172.21.15.79:35360) 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 -- 172.21.15.79:0/2241659363 learned_addr learned my addr 172.21.15.79:0/2241659363 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f6273fff700 1 --2- 172.21.15.79:0/2241659363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f62740ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 -- 172.21.15.79:0/2241659363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 msgr2=0x7f62740ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 --2- 172.21.15.79:0/2241659363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f62740ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 -- 172.21.15.79:0/2241659363 >> v1:172.21.15.156:6789/0 conn(0x7f62740ce930 legacy=0x7f62740af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 -- 172.21.15.79:0/2241659363 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62740b0c80 con 0x7f627415a640 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62788b7700 1 --2- 172.21.15.79:0/2241659363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627415a640 0x7f62740b1480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6268009920 tx=0x7f6268009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993b712db61eff8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62737fe700 1 -- 172.21.15.79:0/2241659363 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268004230 con 0x7f627415a640 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62737fe700 1 -- 172.21.15.79:0/2241659363 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268004390 con 0x7f627415a640 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.569+0000 7f62737fe700 1 -- 172.21.15.79:0/2241659363 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268009e70 con 0x7f627415a640 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 -- 172.21.15.79:0/2241659363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627415a640 msgr2=0x7f62740b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 --2- 172.21.15.79:0/2241659363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627415a640 0x7f62740b1480 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6268009920 tx=0x7f6268009c30 comp rx=0 tx=0).stop 2023-06-08T03:50:48.576 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 -- 172.21.15.79:0/2241659363 shutdown_connections 2023-06-08T03:50:48.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 --2- 172.21.15.79:0/2241659363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f627415a640 0x7f62740b1480 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 --2- 172.21.15.79:0/2241659363 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f62740ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 -- 172.21.15.79:0/2241659363 >> 172.21.15.79:0/2241659363 conn(0x7f627406a670 msgr2=0x7f6274099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:48.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 -- 172.21.15.79:0/2241659363 shutdown_connections 2023-06-08T03:50:48.577 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.570+0000 7f627ab1b700 1 -- 172.21.15.79:0/2241659363 wait complete. 2023-06-08T03:50:48.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.572+0000 7f627ab1b700 1 Processor -- start 2023-06-08T03:50:48.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.572+0000 7f627ab1b700 1 -- start start 2023-06-08T03:50:48.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 0x7f62741581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.579 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f627415a640 0x7f62741e2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f627406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274159100 con 0x7f62740ce930 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274158730 con 0x7f627415a640 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62740b4a00 con 0x7f627415aa10 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 0x7f62741581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 0x7f62741581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35362/0 (socket says 172.21.15.79:35362) 2023-06-08T03:50:48.580 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 -- 172.21.15.79:0/3549046242 learned_addr learned my addr 172.21.15.79:0/3549046242 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f6273fff700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f627415a640 0x7f62741e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62790b8700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f627406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 -- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 msgr2=0x7f627406fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f627406fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 -- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f627415a640 msgr2=0x7f62741e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f627415a640 0x7f62741e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 -- 172.21.15.79:0/3549046242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268009550 con 0x7f62740ce930 2023-06-08T03:50:48.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62790b8700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f627406fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62788b7700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 0x7f62741581f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6268009130 tx=0x7f6268004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800bba0 con 0x7f62740ce930 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626801f070 con 0x7f62740ce930 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62740b3b80 con 0x7f62740ce930 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62680036a0 con 0x7f62740ce930 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.573+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62740b4420 con 0x7f62740ce930 2023-06-08T03:50:48.582 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.574+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62740b3f70 con 0x7f62740ce930 2023-06-08T03:50:48.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.575+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6268021030 con 0x7f62740ce930 2023-06-08T03:50:48.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.575+0000 7f62717fa700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f625806ace0 0x7f625806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.575+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f626808ae30 con 0x7f62740ce930 2023-06-08T03:50:48.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.576+0000 7f6273fff700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f625806ace0 0x7f625806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.576+0000 7f6273fff700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f625806ace0 0x7f625806d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f627406f630 tx=0x7f6260008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.584 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.580+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6268055d00 con 0x7f62740ce930 2023-06-08T03:50:48.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.772+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62740b3f70 con 0x7f62740ce930 2023-06-08T03:50:48.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.774+0000 7f62717fa700 1 -- 172.21.15.79:0/3549046242 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f6268059320 con 0x7f62740ce930 2023-06-08T03:50:48.778 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:48.778 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:48.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f625806ace0 msgr2=0x7f625806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f625806ace0 0x7f625806d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f627406f630 tx=0x7f6260008040 comp rx=0 tx=0).stop 2023-06-08T03:50:48.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 msgr2=0x7f62741581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 0x7f62741581f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6268009130 tx=0x7f6268004750 comp rx=0 tx=0).stop 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 shutdown_connections 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f625806ace0 0x7f625806d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f627415aa10 0x7f627406fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62740ce930 0x7f62741581f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 --2- 172.21.15.79:0/3549046242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f627415a640 0x7f62741e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 >> 172.21.15.79:0/3549046242 conn(0x7f627406a670 msgr2=0x7f62740728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 shutdown_connections 2023-06-08T03:50:48.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.776+0000 7f627ab1b700 1 -- 172.21.15.79:0/3549046242 wait complete. 2023-06-08T03:50:48.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:50:48.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:50:48.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.911+0000 7efe47044700 1 Processor -- start 2023-06-08T03:50:48.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.911+0000 7efe47044700 1 -- start start 2023-06-08T03:50:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe47044700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe40070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe47044700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe40070bd0 con 0x7efe400cce80 2023-06-08T03:50:48.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe47044700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe400cad50 con 0x7efe400cb1e0 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe47044700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400ce570 con 0x7efe4015fb10 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe44de0700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7efe400cce80 0x7efe400cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58846/0 (socket says 172.21.15.79:58846) 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe44de0700 1 -- 172.21.15.79:0/3841189610 learned_addr learned my addr 172.21.15.79:0/3841189610 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710044823 0 0) 0x7efe34001110 con 0x7efe400cce80 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3ffff700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe40070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe300035b0 con 0x7efe400cce80 2023-06-08T03:50:48.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3645306472 0 0) 0x7efe300035b0 con 0x7efe400cce80 2023-06-08T03:50:48.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 msgr2=0x7efe40070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:48.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe40070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 >> v1:172.21.15.156:6789/0 conn(0x7efe400cb1e0 legacy=0x7efe400cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe40160080 con 0x7efe400cce80 2023-06-08T03:50:48.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7efe34001800 con 0x7efe400cce80 2023-06-08T03:50:48.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.912+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 >> v1:172.21.15.79:6789/0 conn(0x7efe400cce80 legacy=0x7efe400cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30003a30 0x7efe30003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe30004850 0x7efe30004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe30005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400ce570 con 0x7efe30004850 2023-06-08T03:50:48.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300142d0 con 0x7efe30003a30 2023-06-08T03:50:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe30014460 con 0x7efe4015fb10 2023-06-08T03:50:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe30004850 0x7efe30004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe44de0700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30003a30 0x7efe30003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe455e1700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe30005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 -- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 msgr2=0x7efe30005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:48.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe30005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 -- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30003a30 msgr2=0x7efe30003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30003a30 0x7efe30003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 -- 172.21.15.79:0/3841189610 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300035b0 con 0x7efe30004850 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe455e1700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe4015fb10 0x7efe30005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3ffff700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe30004850 0x7efe30004cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009b60 tx=0x7efe30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6127082110064a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00e6c0 con 0x7efe30004850 2023-06-08T03:50:48.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c00e820 con 0x7efe30004850 2023-06-08T03:50:48.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.913+0000 7efe3f7fe700 1 -- 172.21.15.79:0/3841189610 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00eb90 con 0x7efe30004850 2023-06-08T03:50:48.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe47044700 1 -- 172.21.15.79:0/3841189610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe30004850 msgr2=0x7efe30004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe47044700 1 --2- 172.21.15.79:0/3841189610 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe30004850 0x7efe30004cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009b60 tx=0x7efe30005880 comp rx=0 tx=0).stop 2023-06-08T03:50:48.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe455e1700 1 -- 172.21.15.79:0/3841189610 reap_dead start 2023-06-08T03:50:48.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe47044700 1 -- 172.21.15.79:0/3841189610 shutdown_connections 2023-06-08T03:50:48.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe47044700 1 -- 172.21.15.79:0/3841189610 >> 172.21.15.79:0/3841189610 conn(0x7efe4006e1e0 msgr2=0x7efe4006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe47044700 1 -- 172.21.15.79:0/3841189610 shutdown_connections 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.914+0000 7efe47044700 1 -- 172.21.15.79:0/3841189610 wait complete. 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.915+0000 7efe47044700 1 Processor -- start 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.915+0000 7efe47044700 1 -- start start 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe47044700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30004850 0x7efe40155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe47044700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe40151bc0 0x7efe4015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe47044700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 0x7efe40143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe47044700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe40155750 con 0x7efe400c99c0 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe47044700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe401528a0 con 0x7efe30004850 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe47044700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b13d0 con 0x7efe40151bc0 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 0x7efe40143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 0x7efe40143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35386/0 (socket says 172.21.15.79:35386) 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 -- 172.21.15.79:0/3311815350 learned_addr learned my addr 172.21.15.79:0/3311815350 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe44de0700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30004850 0x7efe40155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe3ffff700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe40151bc0 0x7efe4015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 -- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe40151bc0 msgr2=0x7efe4015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe40151bc0 0x7efe4015c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 -- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30004850 msgr2=0x7efe40155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30004850 0x7efe40155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 -- 172.21.15.79:0/3311815350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe400b16b0 con 0x7efe400c99c0 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.916+0000 7efe455e1700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 0x7efe40143510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efe400cf2a0 tx=0x7efe3800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.917+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38003c10 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.917+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38003d90 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.917+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe400b1990 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.917+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38005310 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.917+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400b1f50 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.918+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe400b20e0 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.918+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe38005470 con 0x7efe400c99c0 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.918+0000 7efe3d7fa700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efe2406ace0 0x7efe2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:48.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.919+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efe38089730 con 0x7efe400c99c0 2023-06-08T03:50:48.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.919+0000 7efe44de0700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efe2406ace0 0x7efe2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:48.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.919+0000 7efe44de0700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efe2406ace0 0x7efe2406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe34000c00 tx=0x7efe34001d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:48.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:48.923+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efe38054460 con 0x7efe400c99c0 2023-06-08T03:50:49.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.108+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efe40073e90 con 0x7efe400c99c0 2023-06-08T03:50:49.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.108+0000 7efe3d7fa700 1 -- 172.21.15.79:0/3311815350 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7efe38057a80 con 0x7efe400c99c0 2023-06-08T03:50:49.112 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:50:49.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efe2406ace0 msgr2=0x7efe2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efe2406ace0 0x7efe2406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe34000c00 tx=0x7efe34001d80 comp rx=0 tx=0).stop 2023-06-08T03:50:49.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 msgr2=0x7efe40143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 0x7efe40143510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efe400cf2a0 tx=0x7efe3800edc0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 shutdown_connections 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe400c99c0 0x7efe40143510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe30004850 0x7efe40155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efe2406ace0 0x7efe2406d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 --2- 172.21.15.79:0/3311815350 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe40151bc0 0x7efe4015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 >> 172.21.15.79:0/3311815350 conn(0x7efe4006e1e0 msgr2=0x7efe400cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:49.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 shutdown_connections 2023-06-08T03:50:49.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.110+0000 7efe47044700 1 -- 172.21.15.79:0/3311815350 wait complete. 2023-06-08T03:50:49.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:50:49.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.244+0000 7fdd4e51f700 1 Processor -- start 2023-06-08T03:50:49.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.244+0000 7fdd4e51f700 1 -- start start 2023-06-08T03:50:49.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4e51f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd48069d50 0x7fdd4806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4e51f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd4806a120 0x7fdd48071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4e51f700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4815c690 con 0x7fdd480cbbf0 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4e51f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480cef60 con 0x7fdd48069d50 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4e51f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd480cf240 con 0x7fdd4806a120 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd47fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd48069d50 0x7fdd4806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4cabc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdd480cbbf0 0x7fdd480cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58868/0 (socket says 172.21.15.79:58868) 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd4cabc700 1 -- 172.21.15.79:0/3324187416 learned_addr learned my addr 172.21.15.79:0/3324187416 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd477fe700 1 --2- 172.21.15.79:0/3324187416 >> v2:172.21.15.156:3300/0 conn(0x7fdd4806a120 0x7fdd48071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd47fff700 1 -- 172.21.15.79:0/3324187416 >> v2:172.21.15.156:3300/0 conn(0x7fdd4806a120 msgr2=0x7fdd48071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd47fff700 1 --2- 172.21.15.79:0/3324187416 >> v2:172.21.15.156:3300/0 conn(0x7fdd4806a120 0x7fdd48071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd47fff700 1 -- 172.21.15.79:0/3324187416 >> v1:172.21.15.79:6789/0 conn(0x7fdd480cbbf0 legacy=0x7fdd480cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd47fff700 1 -- 172.21.15.79:0/3324187416 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4806eac0 con 0x7fdd48069d50 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.245+0000 7fdd46ffd700 1 -- 172.21.15.79:0/3324187416 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231361114 0 0) 0x7fdd4815c690 con 0x7fdd480cbbf0 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd47fff700 1 --2- 172.21.15.79:0/3324187416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd48069d50 0x7fdd4806f5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd4806cc00 tx=0x7fdd3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd08541567732466 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd46ffd700 1 -- 172.21.15.79:0/3324187416 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c011830 con 0x7fdd48069d50 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd46ffd700 1 -- 172.21.15.79:0/3324187416 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c011990 con 0x7fdd48069d50 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd46ffd700 1 -- 172.21.15.79:0/3324187416 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c011d20 con 0x7fdd48069d50 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 -- 172.21.15.79:0/3324187416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd48069d50 msgr2=0x7fdd4806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/3324187416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd48069d50 0x7fdd4806f5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd4806cc00 tx=0x7fdd3c009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:49.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 -- 172.21.15.79:0/3324187416 shutdown_connections 2023-06-08T03:50:49.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/3324187416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd48069d50 0x7fdd4806f5a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/3324187416 >> v2:172.21.15.156:3300/0 conn(0x7fdd4806a120 0x7fdd48071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 -- 172.21.15.79:0/3324187416 >> 172.21.15.79:0/3324187416 conn(0x7fdd480b1d30 msgr2=0x7fdd4806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:49.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 -- 172.21.15.79:0/3324187416 shutdown_connections 2023-06-08T03:50:49.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.246+0000 7fdd4e51f700 1 -- 172.21.15.79:0/3324187416 wait complete. 2023-06-08T03:50:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.247+0000 7fdd4e51f700 1 Processor -- start 2023-06-08T03:50:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 -- start start 2023-06-08T03:50:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd48069d50 0x7fdd48156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 0x7fdd480cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd480cbbf0 0x7fdd480cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480c8060 con 0x7fdd4806a120 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480c82b0 con 0x7fdd48069d50 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd4e51f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480c8590 con 0x7fdd480cbbf0 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd477fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 0x7fdd480cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd477fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 0x7fdd480cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35394/0 (socket says 172.21.15.79:35394) 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.248+0000 7fdd477fe700 1 -- 172.21.15.79:0/2034314987 learned_addr learned my addr 172.21.15.79:0/2034314987 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd47fff700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd48069d50 0x7fdd48156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd4cabc700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd480cbbf0 0x7fdd480cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd477fe700 1 -- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd480cbbf0 msgr2=0x7fdd480cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd477fe700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd480cbbf0 0x7fdd480cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd477fe700 1 -- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd48069d50 msgr2=0x7fdd48156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd477fe700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd48069d50 0x7fdd48156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd477fe700 1 -- 172.21.15.79:0/2034314987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c0094a0 con 0x7fdd4806a120 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd4cabc700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd480cbbf0 0x7fdd480cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd477fe700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 0x7fdd480cf480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800e9b0 tx=0x7fdd3800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38009710 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3800c850 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd480c8870 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800cbc0 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.249+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd480c9070 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.250+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd48073e90 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.251+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd3800cd20 con 0x7fdd4806a120 2023-06-08T03:50:49.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.251+0000 7fdd44ff9700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd3406ace0 0x7fdd3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.251+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdd38089860 con 0x7fdd4806a120 2023-06-08T03:50:49.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.251+0000 7fdd47fff700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd3406ace0 0x7fdd3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.252+0000 7fdd47fff700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd3406ace0 0x7fdd3406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0090b0 tx=0x7fdd3c01c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.255+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd38054620 con 0x7fdd4806a120 2023-06-08T03:50:49.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.442+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd480aecf0 con 0x7fdd4806a120 2023-06-08T03:50:49.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.443+0000 7fdd44ff9700 1 -- 172.21.15.79:0/2034314987 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd38057c40 con 0x7fdd4806a120 2023-06-08T03:50:49.447 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:50:49.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd3406ace0 msgr2=0x7fdd3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd3406ace0 0x7fdd3406d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0090b0 tx=0x7fdd3c01c040 comp rx=0 tx=0).stop 2023-06-08T03:50:49.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 msgr2=0x7fdd480cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 0x7fdd480cf480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800e9b0 tx=0x7fdd3800ecc0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 shutdown_connections 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd3406ace0 0x7fdd3406d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd480cbbf0 0x7fdd480cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd48069d50 0x7fdd48156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 --2- 172.21.15.79:0/2034314987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd4806a120 0x7fdd480cf480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 >> 172.21.15.79:0/2034314987 conn(0x7fdd480b1d30 msgr2=0x7fdd480cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:49.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 shutdown_connections 2023-06-08T03:50:49.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.445+0000 7fdd4e51f700 1 -- 172.21.15.79:0/2034314987 wait complete. 2023-06-08T03:50:49.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:50:49.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.579+0000 7f6c2de0b700 1 Processor -- start 2023-06-08T03:50:49.583 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.579+0000 7f6c2de0b700 1 -- start start 2023-06-08T03:50:49.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c2de0b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6c2806a460 0x7f6c28071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c2de0b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c280cd140 con 0x7f6c2806d8a0 2023-06-08T03:50:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c2de0b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c28071110 con 0x7f6c2815c1f0 2023-06-08T03:50:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c2de0b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c28071be0 con 0x7f6c2806a460 2023-06-08T03:50:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c26ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6c2806d8a0 0x7f6c28069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58878/0 (socket says 172.21.15.79:58878) 2023-06-08T03:50:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c26ffd700 1 -- 172.21.15.79:0/1734375953 learned_addr learned my addr 172.21.15.79:0/1734375953 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:49.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c27fff700 1 --2- 172.21.15.79:0/1734375953 >> v2:172.21.15.156:3300/0 conn(0x7f6c2806a460 0x7f6c28071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c267fc700 1 -- 172.21.15.79:0/1734375953 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310082468 0 0) 0x7f6c280cd140 con 0x7f6c2806d8a0 2023-06-08T03:50:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c267fc700 1 -- 172.21.15.79:0/1734375953 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c100035b0 con 0x7f6c2806d8a0 2023-06-08T03:50:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c27fff700 1 -- 172.21.15.79:0/1734375953 >> v1:172.21.15.156:6789/0 conn(0x7f6c2815c1f0 legacy=0x7f6c280cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c27fff700 1 -- 172.21.15.79:0/1734375953 >> v1:172.21.15.79:6789/0 conn(0x7f6c2806d8a0 legacy=0x7f6c28069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c267fc700 1 -- 172.21.15.79:0/1734375953 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55104791 0 0) 0x7f6c100035b0 con 0x7f6c2806d8a0 2023-06-08T03:50:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.580+0000 7f6c27fff700 1 -- 172.21.15.79:0/1734375953 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280cd390 con 0x7f6c2806a460 2023-06-08T03:50:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c27fff700 1 --2- 172.21.15.79:0/1734375953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2806a460 0x7f6c28071fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009d90 tx=0x7f6c2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d86fcdefc7dbb server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c267fc700 1 -- 172.21.15.79:0/1734375953 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000e070 con 0x7f6c2806a460 2023-06-08T03:50:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c267fc700 1 -- 172.21.15.79:0/1734375953 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20003a20 con 0x7f6c2806a460 2023-06-08T03:50:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c267fc700 1 -- 172.21.15.79:0/1734375953 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c18001670 con 0x7f6c2806a460 2023-06-08T03:50:49.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 -- 172.21.15.79:0/1734375953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2806a460 msgr2=0x7f6c28071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/1734375953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2806a460 0x7f6c28071fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009d90 tx=0x7f6c2000b330 comp rx=0 tx=0).stop 2023-06-08T03:50:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 -- 172.21.15.79:0/1734375953 shutdown_connections 2023-06-08T03:50:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/1734375953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2806a460 0x7f6c28071fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 -- 172.21.15.79:0/1734375953 >> 172.21.15.79:0/1734375953 conn(0x7f6c280b1d30 msgr2=0x7f6c2806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 -- 172.21.15.79:0/1734375953 shutdown_connections 2023-06-08T03:50:49.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.581+0000 7f6c2de0b700 1 -- 172.21.15.79:0/1734375953 wait complete. 2023-06-08T03:50:49.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 Processor -- start 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 -- start start 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 0x7f6c28150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c2806d8a0 0x7f6c2815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2815c1f0 0x7f6c2815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280c79c0 con 0x7f6c2806a460 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28151060 con 0x7f6c2815c1f0 2023-06-08T03:50:49.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.583+0000 7f6c2de0b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280c6d40 con 0x7f6c2806d8a0 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 0x7f6c28150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 0x7f6c28150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35406/0 (socket says 172.21.15.79:35406) 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 -- 172.21.15.79:0/544319762 learned_addr learned my addr 172.21.15.79:0/544319762 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c27fff700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2815c1f0 0x7f6c2815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 -- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c2806d8a0 msgr2=0x7f6c2815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c2806d8a0 0x7f6c2815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 -- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2815c1f0 msgr2=0x7f6c2815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2815c1f0 0x7f6c2815cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 -- 172.21.15.79:0/544319762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c20009a70 con 0x7f6c2806a460 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c277fe700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 0x7f6c28150b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c450 tx=0x7f6c1c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0052d0 con 0x7f6c2806a460 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c005430 con 0x7f6c2806a460 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c280c7020 con 0x7f6c2806a460 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0038c0 con 0x7f6c2806a460 2023-06-08T03:50:49.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.584+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c280c75e0 con 0x7f6c2806a460 2023-06-08T03:50:49.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.585+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c28072e80 con 0x7f6c2806a460 2023-06-08T03:50:49.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.585+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c1c003a20 con 0x7f6c2806a460 2023-06-08T03:50:49.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.586+0000 7f6c0ffff700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c1406ace0 0x7f6c1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.586+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c1c088a70 con 0x7f6c2806a460 2023-06-08T03:50:49.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.586+0000 7f6c26ffd700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c1406ace0 0x7f6c1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.587+0000 7f6c26ffd700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c1406ace0 0x7f6c1406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c18001eb0 tx=0x7f6c18001980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.590+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c1c053830 con 0x7f6c2806a460 2023-06-08T03:50:49.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.774+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c28073e90 con 0x7f6c2806a460 2023-06-08T03:50:49.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.775+0000 7f6c0ffff700 1 -- 172.21.15.79:0/544319762 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c1c056e50 con 0x7f6c2806a460 2023-06-08T03:50:49.779 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:49.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c1406ace0 msgr2=0x7f6c1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c1406ace0 0x7f6c1406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c18001eb0 tx=0x7f6c18001980 comp rx=0 tx=0).stop 2023-06-08T03:50:49.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 msgr2=0x7f6c28150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 0x7f6c28150b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00c450 tx=0x7f6c1c00c810 comp rx=0 tx=0).stop 2023-06-08T03:50:49.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 shutdown_connections 2023-06-08T03:50:49.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c2815c1f0 0x7f6c2815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c2806a460 0x7f6c28150b20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6c1406ace0 0x7f6c1406d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 --2- 172.21.15.79:0/544319762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c2806d8a0 0x7f6c2815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 >> 172.21.15.79:0/544319762 conn(0x7f6c280b1d30 msgr2=0x7f6c28072770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:49.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 shutdown_connections 2023-06-08T03:50:49.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.777+0000 7f6c2de0b700 1 -- 172.21.15.79:0/544319762 wait complete. 2023-06-08T03:50:49.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:50:49.915 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.911+0000 7f8b3ae47700 1 Processor -- start 2023-06-08T03:50:49.916 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.911+0000 7f8b3ae47700 1 -- start start 2023-06-08T03:50:49.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b3ae47700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b3406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b3ae47700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b340cb1e0 0x7f8b3406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b3ae47700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3415a460 con 0x7f8b340cc900 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b3ae47700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b34069d50 con 0x7f8b340cb1e0 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b3ae47700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34069f60 con 0x7f8b340ae150 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b38be3700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8b340cc900 0x7f8b340cccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58890/0 (socket says 172.21.15.79:58890) 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b38be3700 1 -- 172.21.15.79:0/1161048238 learned_addr learned my addr 172.21.15.79:0/1161048238 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b393e4700 1 --2- 172.21.15.79:0/1161048238 >> v2:172.21.15.156:3300/0 conn(0x7f8b340cb1e0 0x7f8b3406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.912+0000 7f8b33fff700 1 --2- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b3406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b337fe700 1 -- 172.21.15.79:0/1161048238 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220667859 0 0) 0x7f8b3415a460 con 0x7f8b340cc900 2023-06-08T03:50:49.918 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b337fe700 1 -- 172.21.15.79:0/1161048238 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b240035b0 con 0x7f8b340cc900 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b337fe700 1 -- 172.21.15.79:0/1161048238 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832103815 0 0) 0x7f8b240035b0 con 0x7f8b340cc900 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b393e4700 1 -- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 msgr2=0x7f8b3406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b393e4700 1 --2- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b3406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b393e4700 1 -- 172.21.15.79:0/1161048238 >> v1:172.21.15.79:6789/0 conn(0x7f8b340cc900 legacy=0x7f8b340cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b393e4700 1 -- 172.21.15.79:0/1161048238 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3406a240 con 0x7f8b340cb1e0 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b393e4700 1 --2- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b3406aa40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009d90 tx=0x7f8b2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65804a1dfd3571f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b337fe700 1 -- 172.21.15.79:0/1161048238 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00e070 con 0x7f8b340cb1e0 2023-06-08T03:50:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b337fe700 1 -- 172.21.15.79:0/1161048238 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c003a20 con 0x7f8b340cb1e0 2023-06-08T03:50:49.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.913+0000 7f8b337fe700 1 -- 172.21.15.79:0/1161048238 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28001670 con 0x7f8b340cb1e0 2023-06-08T03:50:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 -- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 msgr2=0x7f8b3406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b3406aa40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009d90 tx=0x7f8b2c00b330 comp rx=0 tx=0).stop 2023-06-08T03:50:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 -- 172.21.15.79:0/1161048238 shutdown_connections 2023-06-08T03:50:49.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b3406aa40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/1161048238 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b3406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 -- 172.21.15.79:0/1161048238 >> 172.21.15.79:0/1161048238 conn(0x7f8b340ced30 msgr2=0x7f8b340cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 -- 172.21.15.79:0/1161048238 shutdown_connections 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.914+0000 7f8b3ae47700 1 -- 172.21.15.79:0/1161048238 wait complete. 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.915+0000 7f8b3ae47700 1 Processor -- start 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.915+0000 7f8b3ae47700 1 -- start start 2023-06-08T03:50:49.923 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b3ae47700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b340c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b3ae47700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b340c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b3ae47700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 0x7f8b34158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b3ae47700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c79a0 con 0x7f8b340cc900 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b3ae47700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c6d90 con 0x7f8b340cb1e0 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b3ae47700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340c7070 con 0x7f8b340ae150 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 0x7f8b34158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.924 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 0x7f8b34158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35424/0 (socket says 172.21.15.79:35424) 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 -- 172.21.15.79:0/2962434603 learned_addr learned my addr 172.21.15.79:0/2962434603 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b38be3700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b340c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b33fff700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b340c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 -- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 msgr2=0x7f8b340c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b340c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 -- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 msgr2=0x7f8b340c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b340c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:49.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b393e4700 1 -- 172.21.15.79:0/2962434603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c009a70 con 0x7f8b340cc900 2023-06-08T03:50:49.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.916+0000 7f8b33fff700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b340c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:49.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.917+0000 7f8b393e4700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 0x7f8b34158270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b34066790 tx=0x7f8b2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.917+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c009410 con 0x7f8b340cc900 2023-06-08T03:50:49.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.917+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0044e0 con 0x7f8b340cc900 2023-06-08T03:50:49.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.917+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b340c7350 con 0x7f8b340cc900 2023-06-08T03:50:49.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.917+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c004890 con 0x7f8b340cc900 2023-06-08T03:50:49.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.917+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340c6460 con 0x7f8b340cc900 2023-06-08T03:50:49.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.918+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3415a0f0 con 0x7f8b340cc900 2023-06-08T03:50:49.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.918+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8b2c012430 con 0x7f8b340cc900 2023-06-08T03:50:49.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.918+0000 7f8b317fa700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8b1806ace0 0x7f8b1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:49.927 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.918+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8b2c08b6a0 con 0x7f8b340cc900 2023-06-08T03:50:49.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.919+0000 7f8b38be3700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8b1806ace0 0x7f8b1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:49.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.919+0000 7f8b38be3700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8b1806ace0 0x7f8b1806d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000f60 tx=0x7f8b28001230 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:49.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:49.923+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8b3415a0f0 con 0x7f8b340cc900 2023-06-08T03:50:50.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.110+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b3415a0f0 con 0x7f8b340cc900 2023-06-08T03:50:50.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.110+0000 7f8b317fa700 1 -- 172.21.15.79:0/2962434603 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b3415a0f0 con 0x7f8b340cc900 2023-06-08T03:50:50.114 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:50:50.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8b1806ace0 msgr2=0x7f8b1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8b1806ace0 0x7f8b1806d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000f60 tx=0x7f8b28001230 comp rx=0 tx=0).stop 2023-06-08T03:50:50.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 msgr2=0x7f8b34158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 0x7f8b34158270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b34066790 tx=0x7f8b2c004ad0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 shutdown_connections 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b340cc900 0x7f8b34158270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8b1806ace0 0x7f8b1806d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b340ae150 0x7f8b340c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 --2- 172.21.15.79:0/2962434603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b340cb1e0 0x7f8b340c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 >> 172.21.15.79:0/2962434603 conn(0x7f8b340ced30 msgr2=0x7f8b340af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 shutdown_connections 2023-06-08T03:50:50.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.112+0000 7f8b3ae47700 1 -- 172.21.15.79:0/2962434603 wait complete. 2023-06-08T03:50:50.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:50:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.246+0000 7fa8f818d700 1 Processor -- start 2023-06-08T03:50:50.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.247+0000 7fa8f818d700 1 -- start start 2023-06-08T03:50:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.247+0000 7fa8f818d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 0x7fa8f0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.247+0000 7fa8f818d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00cd140 con 0x7fa8f0069d50 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.247+0000 7fa8f818d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f0071110 con 0x7fa8f015c1f0 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.247+0000 7fa8f818d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0071be0 con 0x7fa8f006d8a0 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.247+0000 7fa8f672a700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa8f0069d50 0x7fa8f00cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58908/0 (socket says 172.21.15.79:58908) 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f672a700 1 -- 172.21.15.79:0/3452278580 learned_addr learned my addr 172.21.15.79:0/3452278580 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5728700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 0x7fa8f0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711586204 0 0) 0x7fa8f00cd140 con 0x7fa8f0069d50 2023-06-08T03:50:50.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8dc0035b0 con 0x7fa8f0069d50 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405153379 0 0) 0x7fa8f0071110 con 0x7fa8f015c1f0 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f00cd140 con 0x7fa8f015c1f0 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 784237425 0 0) 0x7fa8dc0035b0 con 0x7fa8f0069d50 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 msgr2=0x7fa8f0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 0x7fa8f0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 >> v1:172.21.15.156:6789/0 conn(0x7fa8f015c1f0 legacy=0x7fa8f00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00cd390 con 0x7fa8f0069d50 2023-06-08T03:50:50.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fa8e80019f0 con 0x7fa8f0069d50 2023-06-08T03:50:50.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 >> v1:172.21.15.79:6789/0 conn(0x7fa8f0069d50 legacy=0x7fa8f00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8dc003bc0 0x7fa8dc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 0x7fa8dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc004d90 0x7fa8dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0071be0 con 0x7fa8dc003bc0 2023-06-08T03:50:50.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc00bbf0 con 0x7fa8dc004d90 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc00bdb0 con 0x7fa8f006d8a0 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8dc003bc0 0x7fa8dc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5728700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 0x7fa8dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f672a700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc004d90 0x7fa8dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5f29700 1 -- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 msgr2=0x7fa8dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f006d8a0 0x7fa8dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5f29700 1 -- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc004d90 msgr2=0x7fa8dc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc004d90 0x7fa8dc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f5f29700 1 -- 172.21.15.79:0/3452278580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0035b0 con 0x7fa8dc003bc0 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.248+0000 7fa8f672a700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc004d90 0x7fa8dc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8dc003bc0 0x7fa8dc003fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0001940 tx=0x7fa8dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacb9f04513e63a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0001430 con 0x7fa8dc003bc0 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0001d70 con 0x7fa8dc003bc0 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f4f27700 1 -- 172.21.15.79:0/3452278580 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0005510 con 0x7fa8dc003bc0 2023-06-08T03:50:50.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f818d700 1 -- 172.21.15.79:0/3452278580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8dc003bc0 msgr2=0x7fa8dc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f818d700 1 --2- 172.21.15.79:0/3452278580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8dc003bc0 0x7fa8dc003fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0001940 tx=0x7fa8dc005880 comp rx=0 tx=0).stop 2023-06-08T03:50:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f672a700 1 -- 172.21.15.79:0/3452278580 reap_dead start 2023-06-08T03:50:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f818d700 1 -- 172.21.15.79:0/3452278580 shutdown_connections 2023-06-08T03:50:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f818d700 1 -- 172.21.15.79:0/3452278580 >> 172.21.15.79:0/3452278580 conn(0x7fa8f00b1d30 msgr2=0x7fa8f006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f818d700 1 -- 172.21.15.79:0/3452278580 shutdown_connections 2023-06-08T03:50:50.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.249+0000 7fa8f818d700 1 -- 172.21.15.79:0/3452278580 wait complete. 2023-06-08T03:50:50.261 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 Processor -- start 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 -- start start 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc003bc0 0x7fa8f00c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f01f6660 0x7fa8f01f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 0x7fa8f00c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f015ed40 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01f6350 con 0x7fa8dc003bc0 2023-06-08T03:50:50.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f818d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0145210 con 0x7fa8f01f6660 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f672a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 0x7fa8f00c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f672a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 0x7fa8f00c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35442/0 (socket says 172.21.15.79:35442) 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.251+0000 7fa8f672a700 1 -- 172.21.15.79:0/1557978996 learned_addr learned my addr 172.21.15.79:0/1557978996 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f5728700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f01f6660 0x7fa8f01f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc003bc0 0x7fa8f00c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f672a700 1 -- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f01f6660 msgr2=0x7fa8f01f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f672a700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f01f6660 0x7fa8f01f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f672a700 1 -- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc003bc0 msgr2=0x7fa8f00c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f672a700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc003bc0 0x7fa8f00c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f672a700 1 -- 172.21.15.79:0/1557978996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f015e0c0 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc003bc0 0x7fa8f00c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f672a700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 0x7fa8f00c2db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8001330 tx=0x7fa8e800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8002080 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e800ebd0 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f015e3a0 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8013420 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.252+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f015e960 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.253+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f015ee80 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.253+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8e8013580 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.254+0000 7fa8e67fc700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa8d406ace0 0x7fa8d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.254+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa8e8089d80 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.254+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa8d406ace0 0x7fa8d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.255+0000 7fa8f5f29700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa8d406ace0 0x7fa8d406d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0015040 tx=0x7fa8e000a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.258+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8f015ee80 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.448+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa8f015ee80 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.448+0000 7fa8e67fc700 1 -- 172.21.15.79:0/1557978996 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa8f015ee80 con 0x7fa8f01f5eb0 2023-06-08T03:50:50.453 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:50.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa8d406ace0 msgr2=0x7fa8d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa8d406ace0 0x7fa8d406d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0015040 tx=0x7fa8e000a000 comp rx=0 tx=0).stop 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 msgr2=0x7fa8f00c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 0x7fa8f00c2db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8001330 tx=0x7fa8e800e440 comp rx=0 tx=0).stop 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 shutdown_connections 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8f01f5eb0 0x7fa8f00c2db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8dc003bc0 0x7fa8f00c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa8d406ace0 0x7fa8d406d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 --2- 172.21.15.79:0/1557978996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f01f6660 0x7fa8f01f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.450+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 >> 172.21.15.79:0/1557978996 conn(0x7fa8f00b1d30 msgr2=0x7fa8f00b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:50.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.451+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 shutdown_connections 2023-06-08T03:50:50.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.451+0000 7fa8f818d700 1 -- 172.21.15.79:0/1557978996 wait complete. 2023-06-08T03:50:50.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:50.585 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.581+0000 7fc479432700 1 Processor -- start 2023-06-08T03:50:50.586 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.582+0000 7fc479432700 1 -- start start 2023-06-08T03:50:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.582+0000 7fc479432700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc47415c520 0x7fc47406b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.582+0000 7fc479432700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47406a620 con 0x7fc4740d0b50 2023-06-08T03:50:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.582+0000 7fc479432700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47406aea0 con 0x7fc47415c150 2023-06-08T03:50:50.587 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.582+0000 7fc479432700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47406b1f0 con 0x7fc47415c520 2023-06-08T03:50:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4737fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc4740d0b50 0x7fc47406bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58926/0 (socket says 172.21.15.79:58926) 2023-06-08T03:50:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4737fe700 1 -- 172.21.15.79:0/3419172427 learned_addr learned my addr 172.21.15.79:0/3419172427 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4727fc700 1 --2- 172.21.15.79:0/3419172427 >> v2:172.21.15.156:3300/0 conn(0x7fc47415c520 0x7fc47406b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc471ffb700 1 -- 172.21.15.79:0/3419172427 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147722280 0 0) 0x7fc46c001490 con 0x7fc4740d0b50 2023-06-08T03:50:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc471ffb700 1 -- 172.21.15.79:0/3419172427 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4600035b0 con 0x7fc4740d0b50 2023-06-08T03:50:50.588 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc471ffb700 1 -- 172.21.15.79:0/3419172427 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2860943169 0 0) 0x7fc4600035b0 con 0x7fc4740d0b50 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4727fc700 1 -- 172.21.15.79:0/3419172427 >> v1:172.21.15.156:6789/0 conn(0x7fc47415c150 legacy=0x7fc4740d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4727fc700 1 -- 172.21.15.79:0/3419172427 >> v1:172.21.15.79:6789/0 conn(0x7fc4740d0b50 legacy=0x7fc47406bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4727fc700 1 -- 172.21.15.79:0/3419172427 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4740b1c70 con 0x7fc47415c520 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc4727fc700 1 --2- 172.21.15.79:0/3419172427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc47406b610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc464009840 tx=0x7fc464009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0d9c29c8da0623 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc471ffb700 1 -- 172.21.15.79:0/3419172427 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46c001b80 con 0x7fc47415c520 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc471ffb700 1 -- 172.21.15.79:0/3419172427 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc464004030 con 0x7fc47415c520 2023-06-08T03:50:50.589 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.583+0000 7fc471ffb700 1 -- 172.21.15.79:0/3419172427 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4640043c0 con 0x7fc47415c520 2023-06-08T03:50:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 -- 172.21.15.79:0/3419172427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 msgr2=0x7fc47406b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.591 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 --2- 172.21.15.79:0/3419172427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc47406b610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc464009840 tx=0x7fc464009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 -- 172.21.15.79:0/3419172427 shutdown_connections 2023-06-08T03:50:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 --2- 172.21.15.79:0/3419172427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc47406b610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 -- 172.21.15.79:0/3419172427 >> 172.21.15.79:0/3419172427 conn(0x7fc47409bbd0 msgr2=0x7fc4740d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 -- 172.21.15.79:0/3419172427 shutdown_connections 2023-06-08T03:50:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.584+0000 7fc479432700 1 -- 172.21.15.79:0/3419172427 wait complete. 2023-06-08T03:50:50.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.585+0000 7fc479432700 1 Processor -- start 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.585+0000 7fc479432700 1 -- start start 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc479432700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 0x7fc474157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc479432700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc47415c150 0x7fc4740c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc479432700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc4740b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc479432700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4741f1cd0 con 0x7fc4740d0b50 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc479432700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474157590 con 0x7fc47415c520 2023-06-08T03:50:50.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc479432700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4740a03b0 con 0x7fc47415c150 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 0x7fc474157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 0x7fc474157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35456/0 (socket says 172.21.15.79:35456) 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 -- 172.21.15.79:0/2264460248 learned_addr learned my addr 172.21.15.79:0/2264460248 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc4737fe700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc4740b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc4727fc700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc47415c150 0x7fc4740c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 -- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc47415c150 msgr2=0x7fc4740c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc47415c150 0x7fc4740c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 -- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 msgr2=0x7fc4740b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc4740b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc472ffd700 1 -- 172.21.15.79:0/2264460248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640094a0 con 0x7fc4740d0b50 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.586+0000 7fc4727fc700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc47415c150 0x7fc4740c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.587+0000 7fc472ffd700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 0x7fc474157050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00c6b0 tx=0x7fc45c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.587+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0052d0 con 0x7fc4740d0b50 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.587+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c005430 con 0x7fc4740d0b50 2023-06-08T03:50:50.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.587+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47415d5a0 con 0x7fc4740d0b50 2023-06-08T03:50:50.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.587+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0038c0 con 0x7fc4740d0b50 2023-06-08T03:50:50.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.587+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4741f1f50 con 0x7fc4740d0b50 2023-06-08T03:50:50.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.588+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc474073e90 con 0x7fc4740d0b50 2023-06-08T03:50:50.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.588+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc45c003a20 con 0x7fc4740d0b50 2023-06-08T03:50:50.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.588+0000 7fc46b7fe700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc45406ace0 0x7fc45406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.589+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc45c088c30 con 0x7fc4740d0b50 2023-06-08T03:50:50.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.589+0000 7fc4727fc700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc45406ace0 0x7fc45406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.589+0000 7fc4727fc700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc45406ace0 0x7fc45406d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4640090b0 tx=0x7fc464013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.598 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.593+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc45c053960 con 0x7fc4740d0b50 2023-06-08T03:50:50.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.822+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc47406a970 con 0x7fc4740d0b50 2023-06-08T03:50:50.826 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.823+0000 7fc46b7fe700 1 -- 172.21.15.79:0/2264460248 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fc45c056f80 con 0x7fc4740d0b50 2023-06-08T03:50:50.827 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:50.827 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:50:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.824+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc45406ace0 msgr2=0x7fc45406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.824+0000 7fc479432700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc45406ace0 0x7fc45406d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc4640090b0 tx=0x7fc464013040 comp rx=0 tx=0).stop 2023-06-08T03:50:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.824+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 msgr2=0x7fc474157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.824+0000 7fc479432700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 0x7fc474157050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00c6b0 tx=0x7fc45c00c9c0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 shutdown_connections 2023-06-08T03:50:50.829 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc47415c520 0x7fc4740b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4740d0b50 0x7fc474157050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc45406ace0 0x7fc45406d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 --2- 172.21.15.79:0/2264460248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc47415c150 0x7fc4740c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 >> 172.21.15.79:0/2264460248 conn(0x7fc47409bbd0 msgr2=0x7fc47415b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 shutdown_connections 2023-06-08T03:50:50.830 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.825+0000 7fc479432700 1 -- 172.21.15.79:0/2264460248 wait complete. 2023-06-08T03:50:50.831 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:50:50.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:50.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.956+0000 7fe5cb417700 1 Processor -- start 2023-06-08T03:50:50.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5cb417700 1 -- start start 2023-06-08T03:50:50.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5cb417700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 0x7fe5c406f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5cb417700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5c406bb30 0x7fe5c40ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5cb417700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c406edd0 con 0x7fe5c40b0fe0 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5cb417700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c406f7a0 con 0x7fe5c406bb30 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5cb417700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406eaa0 con 0x7fe5c415c680 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5c99b4700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe5c40b0fe0 0x7fe5c415ffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58942/0 (socket says 172.21.15.79:58942) 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.957+0000 7fe5c99b4700 1 -- 172.21.15.79:0/1627611900 learned_addr learned my addr 172.21.15.79:0/1627611900 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/1627611900 >> v2:172.21.15.156:3300/0 conn(0x7fe5c406bb30 0x7fe5c40ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c91b3700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 0x7fe5c406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628598409 0 0) 0x7fe5c406edd0 con 0x7fe5c40b0fe0 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b00035b0 con 0x7fe5c40b0fe0 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248425687 0 0) 0x7fe5b00035b0 con 0x7fe5c40b0fe0 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 msgr2=0x7fe5c406f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 0x7fe5c406f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 >> v2:172.21.15.156:3300/0 conn(0x7fe5c406bb30 msgr2=0x7fe5c40ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 --2- 172.21.15.79:0/1627611900 >> v2:172.21.15.156:3300/0 conn(0x7fe5c406bb30 0x7fe5c40ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4069e80 con 0x7fe5c40b0fe0 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fe5bc0019f0 con 0x7fe5c40b0fe0 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 >> v1:172.21.15.79:6789/0 conn(0x7fe5c40b0fe0 legacy=0x7fe5c415ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b0003a70 0x7fe5b0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 0x7fe5b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406f7a0 con 0x7fe5b0004d90 2023-06-08T03:50:50.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406eaa0 con 0x7fe5b0003a70 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b000baf0 con 0x7fe5c415c680 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/1627611900 >> v2:172.21.15.156:3300/0 conn(0x7fe5c406bb30 0x7fe5c40ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c99b4700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c91b3700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b0003a70 0x7fe5b0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 0x7fe5b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c99b4700 1 -- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 msgr2=0x7fe5b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c99b4700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c415c680 0x7fe5b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c99b4700 1 -- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b0003a70 msgr2=0x7fe5b0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c99b4700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b0003a70 0x7fe5b0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.958+0000 7fe5c99b4700 1 -- 172.21.15.79:0/1627611900 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00035b0 con 0x7fe5b0004d90 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5c99b4700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5b0005200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00a830 tx=0x7fe5b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7636b26e291fe0f2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc012600 con 0x7fe5b0004d90 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc012780 con 0x7fe5b0004d90 2023-06-08T03:50:50.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5bbfff700 1 -- 172.21.15.79:0/1627611900 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc012b30 con 0x7fe5b0004d90 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5cb417700 1 -- 172.21.15.79:0/1627611900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 msgr2=0x7fe5b0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5cb417700 1 --2- 172.21.15.79:0/1627611900 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5b0005200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00a830 tx=0x7fe5b0005880 comp rx=0 tx=0).stop 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5c99b4700 1 -- 172.21.15.79:0/1627611900 reap_dead start 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5cb417700 1 -- 172.21.15.79:0/1627611900 shutdown_connections 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5cb417700 1 -- 172.21.15.79:0/1627611900 >> 172.21.15.79:0/1627611900 conn(0x7fe5c40cbfe0 msgr2=0x7fe5c40cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5cb417700 1 -- 172.21.15.79:0/1627611900 shutdown_connections 2023-06-08T03:50:50.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.959+0000 7fe5cb417700 1 -- 172.21.15.79:0/1627611900 wait complete. 2023-06-08T03:50:50.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.961+0000 7fe5cb417700 1 Processor -- start 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.961+0000 7fe5cb417700 1 -- start start 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5cb417700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5c415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5cb417700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c41f6240 0x7fe5c40ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5cb417700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40ade10 0x7fe5c41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5cb417700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40ae280 con 0x7fe5b0004d90 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5cb417700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40b2390 con 0x7fe5c41f6240 2023-06-08T03:50:50.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5cb417700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c415d3a0 con 0x7fe5c40ade10 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5c415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5c415cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35474/0 (socket says 172.21.15.79:35474) 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 -- 172.21.15.79:0/3573713001 learned_addr learned my addr 172.21.15.79:0/3573713001 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c41f6240 0x7fe5c40ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c99b4700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40ade10 0x7fe5c41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 -- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40ade10 msgr2=0x7fe5c41f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40ade10 0x7fe5c41f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 -- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c41f6240 msgr2=0x7fe5c40ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c41f6240 0x7fe5c40ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 -- 172.21.15.79:0/3573713001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c415d680 con 0x7fe5b0004d90 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c41f6240 0x7fe5c40ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.962+0000 7fe5c91b3700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5c415cac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40119a0 tx=0x7fe5b4011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.963+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400f940 con 0x7fe5b0004d90 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.963+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b400faa0 con 0x7fe5b0004d90 2023-06-08T03:50:50.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.963+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c415d960 con 0x7fe5b0004d90 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.963+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400fe30 con 0x7fe5b0004d90 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.963+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c415d000 con 0x7fe5b0004d90 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.964+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4073e90 con 0x7fe5b0004d90 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.964+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5b4014440 con 0x7fe5b0004d90 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.965+0000 7fe5b9ffb700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5a806ace0 0x7fe5a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.965+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe5b4089da0 con 0x7fe5b0004d90 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.965+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5a806ace0 0x7fe5a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:50.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.966+0000 7fe5c89b2700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5a806ace0 0x7fe5a806d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe5c415e5d0 tx=0x7fe5c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:50.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:50.969+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5b4058680 con 0x7fe5b0004d90 2023-06-08T03:50:51.164 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.160+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5c40b54e0 con 0x7fe5b0004d90 2023-06-08T03:50:51.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.161+0000 7fe5b9ffb700 1 -- 172.21.15.79:0/3573713001 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fe5b4022090 con 0x7fe5b0004d90 2023-06-08T03:50:51.166 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:51.166 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:51.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5a806ace0 msgr2=0x7fe5a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5a806ace0 0x7fe5a806d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe5c415e5d0 tx=0x7fe5c0008040 comp rx=0 tx=0).stop 2023-06-08T03:50:51.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 msgr2=0x7fe5c415cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5c415cac0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40119a0 tx=0x7fe5b4011d60 comp rx=0 tx=0).stop 2023-06-08T03:50:51.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 shutdown_connections 2023-06-08T03:50:51.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5a806ace0 0x7fe5a806d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40ade10 0x7fe5c41f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5b0004d90 0x7fe5c415cac0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 --2- 172.21.15.79:0/3573713001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c41f6240 0x7fe5c40ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.163+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 >> 172.21.15.79:0/3573713001 conn(0x7fe5c40cbfe0 msgr2=0x7fe5c40715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:51.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.164+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 shutdown_connections 2023-06-08T03:50:51.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.164+0000 7fe5cb417700 1 -- 172.21.15.79:0/3573713001 wait complete. 2023-06-08T03:50:51.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:50:51.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:50:51.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.299+0000 7ff88623e700 1 Processor -- start 2023-06-08T03:50:51.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff88623e700 1 -- start start 2023-06-08T03:50:51.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff88623e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8800d0060 0x7ff8800af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff88623e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 0x7ff8800cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff88623e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8800cae50 con 0x7ff8800b20f0 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff88623e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8800cbea0 con 0x7ff8800d0060 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff88623e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800cda30 con 0x7ff8800cd080 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff87ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff8800b20f0 0x7ff8800b2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58960/0 (socket says 172.21.15.79:58960) 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.300+0000 7ff87ffff700 1 -- 172.21.15.79:0/3514482600 learned_addr learned my addr 172.21.15.79:0/3514482600 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493894997 0 0) 0x7ff8800cae50 con 0x7ff8800b20f0 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/3514482600 >> v2:172.21.15.156:3300/0 conn(0x7ff8800d0060 0x7ff8800af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff86c0035b0 con 0x7ff8800b20f0 2023-06-08T03:50:51.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87effd700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 0x7ff8800cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 274460371 0 0) 0x7ff86c0035b0 con 0x7ff8800b20f0 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 msgr2=0x7ff8800cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 0x7ff8800cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 >> v2:172.21.15.156:3300/0 conn(0x7ff8800d0060 msgr2=0x7ff8800af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 --2- 172.21.15.79:0/3514482600 >> v2:172.21.15.156:3300/0 conn(0x7ff8800d0060 0x7ff8800af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88015d930 con 0x7ff8800b20f0 2023-06-08T03:50:51.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7ff8780019f0 con 0x7ff8800b20f0 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/3514482600 >> v2:172.21.15.156:3300/0 conn(0x7ff8800d0060 0x7ff8800af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 >> v1:172.21.15.79:6789/0 conn(0x7ff8800b20f0 legacy=0x7ff8800b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff86c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff86c004850 0x7ff86c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 0x7ff86c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8800cda30 con 0x7ff86c003a00 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00fe50 con 0x7ff86c004850 2023-06-08T03:50:51.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c010010 con 0x7ff8800cd080 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff86c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87effd700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff86c004850 0x7ff86c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 -- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 msgr2=0x7ff86c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800cd080 0x7ff86c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 -- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff86c004850 msgr2=0x7ff86c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff86c004850 0x7ff86c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.301+0000 7ff87f7fe700 1 -- 172.21.15.79:0/3514482600 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c0035b0 con 0x7ff86c003a00 2023-06-08T03:50:51.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff86c003e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff868009390 tx=0x7ff86c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd74395dbd0b444 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868019070 con 0x7ff86c003a00 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868009bf0 con 0x7ff86c003a00 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff87e7fc700 1 -- 172.21.15.79:0/3514482600 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8680128e0 con 0x7ff86c003a00 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff88623e700 1 -- 172.21.15.79:0/3514482600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 msgr2=0x7ff86c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff88623e700 1 --2- 172.21.15.79:0/3514482600 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff86c003e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff868009390 tx=0x7ff86c005880 comp rx=0 tx=0).stop 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff87ffff700 1 -- 172.21.15.79:0/3514482600 reap_dead start 2023-06-08T03:50:51.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff88623e700 1 -- 172.21.15.79:0/3514482600 shutdown_connections 2023-06-08T03:50:51.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff88623e700 1 -- 172.21.15.79:0/3514482600 >> 172.21.15.79:0/3514482600 conn(0x7ff88006e1e0 msgr2=0x7ff88006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:51.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff88623e700 1 -- 172.21.15.79:0/3514482600 shutdown_connections 2023-06-08T03:50:51.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.302+0000 7ff88623e700 1 -- 172.21.15.79:0/3514482600 wait complete. 2023-06-08T03:50:51.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 Processor -- start 2023-06-08T03:50:51.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 -- start start 2023-06-08T03:50:51.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff8801514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800c88f0 0x7ff88015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8800c8d60 0x7ff88015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8801519e0 con 0x7ff86c003a00 2023-06-08T03:50:51.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8801582a0 con 0x7ff8800c8d60 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.304+0000 7ff88623e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff880158580 con 0x7ff8800c88f0 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff8801514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff8801514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35494/0 (socket says 172.21.15.79:35494) 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 -- 172.21.15.79:0/2265642751 learned_addr learned my addr 172.21.15.79:0/2265642751 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 -- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800c88f0 msgr2=0x7ff88015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87effd700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800c88f0 0x7ff88015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800c88f0 0x7ff88015c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 -- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8800c8d60 msgr2=0x7ff88015ca40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8800c8d60 0x7ff88015ca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 -- 172.21.15.79:0/2265642751 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8800c4220 con 0x7ff86c003a00 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff87f7fe700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff8801514a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00fe20 tx=0x7ff8680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8680146a0 con 0x7ff86c003a00 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868027440 con 0x7ff86c003a00 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8800c4500 con 0x7ff86c003a00 2023-06-08T03:50:51.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868014840 con 0x7ff86c003a00 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.305+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8800c4d00 con 0x7ff86c003a00 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.306+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff880073e90 con 0x7ff86c003a00 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.306+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8680277f0 con 0x7ff86c003a00 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.307+0000 7ff877fff700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff86006ace0 0x7ff86006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.307+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff868092920 con 0x7ff86c003a00 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.307+0000 7ff87effd700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff86006ace0 0x7ff86006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.308+0000 7ff87effd700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff86006ace0 0x7ff86006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8801590e0 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.311+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff86805d770 con 0x7ff86c003a00 2023-06-08T03:50:51.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.496+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff880070120 con 0x7ff86c003a00 2023-06-08T03:50:51.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.497+0000 7ff877fff700 1 -- 172.21.15.79:0/2265642751 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff868006cc0 con 0x7ff86c003a00 2023-06-08T03:50:51.501 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:50:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff86006ace0 msgr2=0x7ff86006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff86006ace0 0x7ff86006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8801590e0 tx=0x7ff870008040 comp rx=0 tx=0).stop 2023-06-08T03:50:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 msgr2=0x7ff8801514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff8801514a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00fe20 tx=0x7ff8680047c0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 shutdown_connections 2023-06-08T03:50:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8800c8d60 0x7ff88015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c003a00 0x7ff8801514a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff86006ace0 0x7ff86006d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 --2- 172.21.15.79:0/2265642751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8800c88f0 0x7ff88015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 >> 172.21.15.79:0/2265642751 conn(0x7ff88006e1e0 msgr2=0x7ff88006b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 shutdown_connections 2023-06-08T03:50:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.499+0000 7ff88623e700 1 -- 172.21.15.79:0/2265642751 wait complete. 2023-06-08T03:50:51.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:50:51.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.633+0000 7f2a1d614700 1 Processor -- start 2023-06-08T03:50:51.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.633+0000 7f2a1d614700 1 -- start start 2023-06-08T03:50:51.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a1d614700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 0x7f2a180d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a1d614700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2a180ceb40 0x7f2a180d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a1d614700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a180cdaa0 con 0x7f2a180b1990 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a1d614700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180cdd80 con 0x7f2a180ceb40 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a1d614700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180ce060 con 0x7f2a1815c280 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a177fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2a180b1990 0x7f2a1815fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58980/0 (socket says 172.21.15.79:58980) 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a177fe700 1 -- 172.21.15.79:0/79930598 learned_addr learned my addr 172.21.15.79:0/79930598 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a167fc700 1 --2- 172.21.15.79:0/79930598 >> v2:172.21.15.156:3300/0 conn(0x7f2a180ceb40 0x7f2a180d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a16ffd700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 0x7f2a180d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031073906 0 0) 0x7f2a180cdaa0 con 0x7f2a180b1990 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a000035b0 con 0x7f2a180b1990 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713465772 0 0) 0x7f2a000035b0 con 0x7f2a180b1990 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 msgr2=0x7f2a180d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 0x7f2a180d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 >> v2:172.21.15.156:3300/0 conn(0x7f2a180ceb40 msgr2=0x7f2a180d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 --2- 172.21.15.79:0/79930598 >> v2:172.21.15.156:3300/0 conn(0x7f2a180ceb40 0x7f2a180d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a180ce340 con 0x7f2a180b1990 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f2a100019f0 con 0x7f2a180b1990 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a167fc700 1 --2- 172.21.15.79:0/79930598 >> v2:172.21.15.156:3300/0 conn(0x7f2a180ceb40 0x7f2a180d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 >> v1:172.21.15.79:6789/0 conn(0x7f2a180b1990 legacy=0x7f2a1815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.634+0000 7f2a15ffb700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a00003a00 0x7f2a00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 0x7f2a00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180cdd80 con 0x7f2a00004d90 2023-06-08T03:50:51.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180ce060 con 0x7f2a00003a00 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0000ba10 con 0x7f2a1815c280 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a16ffd700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a00003a00 0x7f2a00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a167fc700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 0x7f2a00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 -- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 msgr2=0x7f2a00004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a1815c280 0x7f2a00004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 -- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a00003a00 msgr2=0x7f2a00003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a00003a00 0x7f2a00003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 -- 172.21.15.79:0/79930598 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a000035b0 con 0x7f2a00004d90 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a177fe700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000a830 tx=0x7f2a00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58cdf3810f4f59a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10012600 con 0x7f2a00004d90 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a100127a0 con 0x7f2a00004d90 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.635+0000 7f2a15ffb700 1 -- 172.21.15.79:0/79930598 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10012b50 con 0x7f2a00004d90 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a1d614700 1 -- 172.21.15.79:0/79930598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a00004d90 msgr2=0x7f2a00005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a1d614700 1 --2- 172.21.15.79:0/79930598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a00004d90 0x7f2a00005200 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000a830 tx=0x7f2a00005880 comp rx=0 tx=0).stop 2023-06-08T03:50:51.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a177fe700 1 -- 172.21.15.79:0/79930598 reap_dead start 2023-06-08T03:50:51.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a1d614700 1 -- 172.21.15.79:0/79930598 shutdown_connections 2023-06-08T03:50:51.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a1d614700 1 -- 172.21.15.79:0/79930598 >> 172.21.15.79:0/79930598 conn(0x7f2a1809bc40 msgr2=0x7f2a180d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:51.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a1d614700 1 -- 172.21.15.79:0/79930598 shutdown_connections 2023-06-08T03:50:51.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.636+0000 7f2a1d614700 1 -- 172.21.15.79:0/79930598 wait complete. 2023-06-08T03:50:51.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.637+0000 7f2a1d614700 1 Processor -- start 2023-06-08T03:50:51.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.637+0000 7f2a1d614700 1 -- start start 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.638+0000 7f2a1d614700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a18156790 0x7f2a1815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.638+0000 7f2a1d614700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a1815f4b0 0x7f2a1806af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.638+0000 7f2a1d614700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 0x7f2a1806b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.638+0000 7f2a1d614700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1815a670 con 0x7f2a180caa10 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.638+0000 7f2a1d614700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a18156b60 con 0x7f2a1815f4b0 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.638+0000 7f2a1d614700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c9970 con 0x7f2a18156790 2023-06-08T03:50:51.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 0x7f2a1806b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 0x7f2a1806b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35516/0 (socket says 172.21.15.79:35516) 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 -- 172.21.15.79:0/466705044 learned_addr learned my addr 172.21.15.79:0/466705044 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a167fc700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a1815f4b0 0x7f2a1806af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a16ffd700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a18156790 0x7f2a1815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 -- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a18156790 msgr2=0x7f2a1815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a18156790 0x7f2a1815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 -- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a1815f4b0 msgr2=0x7f2a1806af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a1815f4b0 0x7f2a1806af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 -- 172.21.15.79:0/466705044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a180c9c50 con 0x7f2a180caa10 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a167fc700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a1815f4b0 0x7f2a1806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a177fe700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 0x7f2a1806b470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a10001cb0 tx=0x7f2a10001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10012610 con 0x7f2a180caa10 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a100017b0 con 0x7f2a180caa10 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a180c9f30 con 0x7f2a180caa10 2023-06-08T03:50:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1001d420 con 0x7f2a180caa10 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.639+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a180ca4f0 con 0x7f2a180caa10 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.640+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a18073e90 con 0x7f2a180caa10 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.641+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a1001d580 con 0x7f2a180caa10 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.641+0000 7f29ff7fe700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a0406ace0 0x7f2a0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.641+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a1008acf0 con 0x7f2a180caa10 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.641+0000 7f2a16ffd700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a0406ace0 0x7f2a0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.642+0000 7f2a16ffd700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a0406ace0 0x7f2a0406d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009a80 tx=0x7f2a0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.645+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a10055ac0 con 0x7f2a180caa10 2023-06-08T03:50:51.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.831+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a180ca6b0 con 0x7f2a180caa10 2023-06-08T03:50:51.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.831+0000 7f29ff7fe700 1 -- 172.21.15.79:0/466705044 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a100590e0 con 0x7f2a180caa10 2023-06-08T03:50:51.835 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:50:51.837 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a0406ace0 msgr2=0x7f2a0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a0406ace0 0x7f2a0406d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009a80 tx=0x7f2a0c008040 comp rx=0 tx=0).stop 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 msgr2=0x7f2a1806b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 0x7f2a1806b470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a10001cb0 tx=0x7f2a10001d80 comp rx=0 tx=0).stop 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 shutdown_connections 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2a180caa10 0x7f2a1806b470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a0406ace0 0x7f2a0406d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a18156790 0x7f2a1815e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 --2- 172.21.15.79:0/466705044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a1815f4b0 0x7f2a1806af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 >> 172.21.15.79:0/466705044 conn(0x7f2a1809bc40 msgr2=0x7f2a180b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:51.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 shutdown_connections 2023-06-08T03:50:51.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.833+0000 7f2a1d614700 1 -- 172.21.15.79:0/466705044 wait complete. 2023-06-08T03:50:51.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:50:51.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.968+0000 7f3ecd143700 1 Processor -- start 2023-06-08T03:50:51.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.968+0000 7f3ecd143700 1 -- start start 2023-06-08T03:50:51.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ecd143700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3ec80707d0 0x7f3ec8070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ecd143700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec8071290 con 0x7f3ec815a0e0 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ecd143700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec8071570 con 0x7f3ec80cacc0 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ecd143700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec80ca4c0 con 0x7f3ec80707d0 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec659c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3ec80707d0 0x7f3ec8070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec6d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3ec815a0e0 0x7f3ec8069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:58998/0 (socket says 172.21.15.79:58998) 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec6d9d700 1 -- 172.21.15.79:0/1498370619 learned_addr learned my addr 172.21.15.79:0/1498370619 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:51.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec659c700 1 -- 172.21.15.79:0/1498370619 >> v1:172.21.15.156:6789/0 conn(0x7f3ec80cacc0 legacy=0x7f3ec806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec659c700 1 -- 172.21.15.79:0/1498370619 >> v1:172.21.15.79:6789/0 conn(0x7f3ec815a0e0 legacy=0x7f3ec8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec5d9b700 1 -- 172.21.15.79:0/1498370619 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840766105 0 0) 0x7f3ec8071290 con 0x7f3ec815a0e0 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.969+0000 7f3ec659c700 1 -- 172.21.15.79:0/1498370619 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80ca7a0 con 0x7f3ec80707d0 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ec659c700 1 --2- 172.21.15.79:0/1498370619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80707d0 0x7f3ec8070c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009840 tx=0x7f3eb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92b0d2d6e32d25e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ec5d9b700 1 -- 172.21.15.79:0/1498370619 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8011830 con 0x7f3ec80707d0 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ec5d9b700 1 -- 172.21.15.79:0/1498370619 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb8011990 con 0x7f3ec80707d0 2023-06-08T03:50:51.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ec5d9b700 1 -- 172.21.15.79:0/1498370619 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8011d20 con 0x7f3ec80707d0 2023-06-08T03:50:51.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 -- 172.21.15.79:0/1498370619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80707d0 msgr2=0x7f3ec8070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 --2- 172.21.15.79:0/1498370619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80707d0 0x7f3ec8070c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009840 tx=0x7f3eb8009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 -- 172.21.15.79:0/1498370619 shutdown_connections 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 --2- 172.21.15.79:0/1498370619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80707d0 0x7f3ec8070c50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 -- 172.21.15.79:0/1498370619 >> 172.21.15.79:0/1498370619 conn(0x7f3ec80ced30 msgr2=0x7f3ec80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 -- 172.21.15.79:0/1498370619 shutdown_connections 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.970+0000 7f3ecd143700 1 -- 172.21.15.79:0/1498370619 wait complete. 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.972+0000 7f3ecd143700 1 Processor -- start 2023-06-08T03:50:51.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.972+0000 7f3ecd143700 1 -- start start 2023-06-08T03:50:51.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ecd143700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec80707d0 0x7f3ec8157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ecd143700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 0x7f3ec815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ecd143700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ec815a0e0 0x7f3ec81f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ecd143700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec81554f0 con 0x7f3ec80cacc0 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ecd143700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec815b6a0 con 0x7f3ec80707d0 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ecd143700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8158250 con 0x7f3ec815a0e0 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ec659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 0x7f3ec815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ec659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 0x7f3ec815b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35528/0 (socket says 172.21.15.79:35528) 2023-06-08T03:50:51.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.973+0000 7f3ec659c700 1 -- 172.21.15.79:0/3776547279 learned_addr learned my addr 172.21.15.79:0/3776547279 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec6d9d700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec80707d0 0x7f3ec8157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec759e700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ec815a0e0 0x7f3ec81f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec659c700 1 -- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ec815a0e0 msgr2=0x7f3ec81f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec659c700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ec815a0e0 0x7f3ec81f1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec659c700 1 -- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec80707d0 msgr2=0x7f3ec8157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec659c700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec80707d0 0x7f3ec8157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:51.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec659c700 1 -- 172.21.15.79:0/3776547279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80094a0 con 0x7f3ec80cacc0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec759e700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ec815a0e0 0x7f3ec81f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ec659c700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 0x7f3ec815b160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8066790 tx=0x7f3eb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb801a8c0 con 0x7f3ec80cacc0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb801aa20 con 0x7f3ec80cacc0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec80b2e10 con 0x7f3ec80cacc0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb801ad90 con 0x7f3ec80cacc0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.974+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec80b33d0 con 0x7f3ec80cacc0 2023-06-08T03:50:51.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.975+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec8156360 con 0x7f3ec80cacc0 2023-06-08T03:50:51.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.975+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3eb8022070 con 0x7f3ec80cacc0 2023-06-08T03:50:51.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.976+0000 7f3eaf7fe700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3eb406ace0 0x7f3eb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:51.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.976+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3eb808a8e0 con 0x7f3ec80cacc0 2023-06-08T03:50:51.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.976+0000 7f3ec6d9d700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3eb406ace0 0x7f3eb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:51.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.976+0000 7f3ec6d9d700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3eb406ace0 0x7f3eb406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000f60 tx=0x7f3ebc007d90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:51.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:51.980+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3eb80557e0 con 0x7f3ec80cacc0 2023-06-08T03:50:52.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.168+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ec80af670 con 0x7f3ec80cacc0 2023-06-08T03:50:52.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.168+0000 7f3eaf7fe700 1 -- 172.21.15.79:0/3776547279 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3eb8058e00 con 0x7f3ec80cacc0 2023-06-08T03:50:52.173 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.170+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3eb406ace0 msgr2=0x7f3eb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.170+0000 7f3ecd143700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3eb406ace0 0x7f3eb406d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc000f60 tx=0x7f3ebc007d90 comp rx=0 tx=0).stop 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 msgr2=0x7f3ec815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 0x7f3ec815b160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8066790 tx=0x7f3eb8004750 comp rx=0 tx=0).stop 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 shutdown_connections 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3eb406ace0 0x7f3eb406d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ec815a0e0 0x7f3ec81f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ec80707d0 0x7f3ec8157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 --2- 172.21.15.79:0/3776547279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ec80cacc0 0x7f3ec815b160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 >> 172.21.15.79:0/3776547279 conn(0x7f3ec80ced30 msgr2=0x7f3ec8158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:52.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 shutdown_connections 2023-06-08T03:50:52.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.171+0000 7f3ecd143700 1 -- 172.21.15.79:0/3776547279 wait complete. 2023-06-08T03:50:52.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:50:52.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.304+0000 7fb441a74700 1 Processor -- start 2023-06-08T03:50:52.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.304+0000 7fb441a74700 1 -- start start 2023-06-08T03:50:52.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb441a74700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c15c1f0 0x7fb43c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb441a74700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb43c071fe0 0x7fb43c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb441a74700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb43c0cf130 0x7fb43c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb441a74700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43c0cd140 con 0x7fb43c0cf130 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb441a74700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43c071110 con 0x7fb43c071fe0 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb441a74700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c071be0 con 0x7fb43c15c1f0 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb43c0cf130 0x7fb43c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb43c0cf130 0x7fb43c069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35538/0 (socket says 172.21.15.79:35538) 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 -- 172.21.15.79:0/1076650336 learned_addr learned my addr 172.21.15.79:0/1076650336 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43affd700 1 --2- 172.21.15.79:0/1076650336 >> v2:172.21.15.156:3300/0 conn(0x7fb43c071fe0 0x7fb43c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43b7fe700 1 --2- 172.21.15.79:0/1076650336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c15c1f0 0x7fb43c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 -- 172.21.15.79:0/1076650336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c15c1f0 msgr2=0x7fb43c06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 --2- 172.21.15.79:0/1076650336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c15c1f0 0x7fb43c06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 -- 172.21.15.79:0/1076650336 >> v2:172.21.15.156:3300/0 conn(0x7fb43c071fe0 msgr2=0x7fb43c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 --2- 172.21.15.79:0/1076650336 >> v2:172.21.15.156:3300/0 conn(0x7fb43c071fe0 0x7fb43c0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 -- 172.21.15.79:0/1076650336 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c0cd390 con 0x7fb43c0cf130 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43affd700 1 --2- 172.21.15.79:0/1076650336 >> v2:172.21.15.156:3300/0 conn(0x7fb43c071fe0 0x7fb43c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:52.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.305+0000 7fb43bfff700 1 --2- 172.21.15.79:0/1076650336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c069d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0707a0 tx=0x7fb43400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ca31b24fe328a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb43a7fc700 1 -- 172.21.15.79:0/1076650336 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400e070 con 0x7fb43c0cf130 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb43a7fc700 1 -- 172.21.15.79:0/1076650336 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43400b910 con 0x7fb43c0cf130 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb43a7fc700 1 -- 172.21.15.79:0/1076650336 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43400bcc0 con 0x7fb43c0cf130 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 -- 172.21.15.79:0/1076650336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 msgr2=0x7fb43c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 --2- 172.21.15.79:0/1076650336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c069d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0707a0 tx=0x7fb43400b330 comp rx=0 tx=0).stop 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 -- 172.21.15.79:0/1076650336 shutdown_connections 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 --2- 172.21.15.79:0/1076650336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c069d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 --2- 172.21.15.79:0/1076650336 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c15c1f0 0x7fb43c06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 --2- 172.21.15.79:0/1076650336 >> v2:172.21.15.156:3300/0 conn(0x7fb43c071fe0 0x7fb43c0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 -- 172.21.15.79:0/1076650336 >> 172.21.15.79:0/1076650336 conn(0x7fb43c0b1d30 msgr2=0x7fb43c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:52.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 -- 172.21.15.79:0/1076650336 shutdown_connections 2023-06-08T03:50:52.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.306+0000 7fb441a74700 1 -- 172.21.15.79:0/1076650336 wait complete. 2023-06-08T03:50:52.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.307+0000 7fb441a74700 1 Processor -- start 2023-06-08T03:50:52.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.307+0000 7fb441a74700 1 -- start start 2023-06-08T03:50:52.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb441a74700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c071fe0 0x7fb43c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb441a74700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.316 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb441a74700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb43c15c1f0 0x7fb43c0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb441a74700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0c3ad0 con 0x7fb43c0cf130 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb441a74700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c1f2190 con 0x7fb43c15c1f0 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb441a74700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c151ea0 con 0x7fb43c071fe0 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb43affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb43affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c0b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35542/0 (socket says 172.21.15.79:35542) 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.308+0000 7fb43affd700 1 -- 172.21.15.79:0/3726815133 learned_addr learned my addr 172.21.15.79:0/3726815133 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:52.317 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb43affd700 1 -- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c071fe0 msgr2=0x7fb43c1513b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb43affd700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c071fe0 0x7fb43c1513b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb43affd700 1 -- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb43c15c1f0 msgr2=0x7fb43c0b4570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb43affd700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb43c15c1f0 0x7fb43c0b4570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb43affd700 1 -- 172.21.15.79:0/3726815133 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434009b30 con 0x7fb43c0cf130 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb43affd700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c0b4030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d800 tx=0x7fb42c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c004030 con 0x7fb43c0cf130 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c004190 con 0x7fb43c0cf130 2023-06-08T03:50:52.318 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c0c2e50 con 0x7fb43c0cf130 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c004520 con 0x7fb43c0cf130 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.309+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c0c3410 con 0x7fb43c0cf130 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.310+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c0c3d40 con 0x7fb43c0cf130 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.310+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb42c0036a0 con 0x7fb43c0cf130 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.311+0000 7fb423fff700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb42806ace0 0x7fb42806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.311+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb42c088f50 con 0x7fb43c0cf130 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.311+0000 7fb43b7fe700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb42806ace0 0x7fb42806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.319 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.311+0000 7fb43b7fe700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb42806ace0 0x7fb42806d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4300099d0 tx=0x7fb430009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.320 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.315+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb42c0578e0 con 0x7fb43c0cf130 2023-06-08T03:50:52.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.500+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb43c073e90 con 0x7fb43c0cf130 2023-06-08T03:50:52.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.501+0000 7fb423fff700 1 -- 172.21.15.79:0/3726815133 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb42c0190a0 con 0x7fb43c0cf130 2023-06-08T03:50:52.505 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:50:52.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb42806ace0 msgr2=0x7fb42806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb42806ace0 0x7fb42806d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb4300099d0 tx=0x7fb430009210 comp rx=0 tx=0).stop 2023-06-08T03:50:52.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 msgr2=0x7fb43c0b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c0b4030 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00d800 tx=0x7fb42c00dbc0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 shutdown_connections 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb43c15c1f0 0x7fb43c0b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb42806ace0 0x7fb42806d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb43c071fe0 0x7fb43c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 --2- 172.21.15.79:0/3726815133 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb43c0cf130 0x7fb43c0b4030 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 >> 172.21.15.79:0/3726815133 conn(0x7fb43c0b1d30 msgr2=0x7fb43c15cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 shutdown_connections 2023-06-08T03:50:52.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.503+0000 7fb441a74700 1 -- 172.21.15.79:0/3726815133 wait complete. 2023-06-08T03:50:52.518 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:50:52.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 Processor -- start 2023-06-08T03:50:52.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 -- start start 2023-06-08T03:50:52.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f280406d8a0 0x7f2804071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2804069d50 0x7f280406a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28040cd140 con 0x7f280415c1f0 2023-06-08T03:50:52.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280406a730 con 0x7f280406d8a0 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.638+0000 7f280b067700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2804071110 con 0x7f2804069d50 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f280406d8a0 0x7f2804071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f280406d8a0 0x7f2804071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35550/0 (socket says 172.21.15.79:35550) 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 -- 172.21.15.79:0/3111987277 learned_addr learned my addr 172.21.15.79:0/3111987277 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2809604700 1 --2- 172.21.15.79:0/3111987277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2804069d50 0x7f280406a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 -- 172.21.15.79:0/3111987277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2804069d50 msgr2=0x7f280406a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:52.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 --2- 172.21.15.79:0/3111987277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2804069d50 0x7f280406a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 -- 172.21.15.79:0/3111987277 >> v1:172.21.15.156:6789/0 conn(0x7f280415c1f0 legacy=0x7f28040cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 -- 172.21.15.79:0/3111987277 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804071be0 con 0x7f280406d8a0 2023-06-08T03:50:52.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f2803fff700 1 --2- 172.21.15.79:0/3111987277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f280406d8a0 0x7f2804071fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098f0 tx=0x7f27f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb471353e17355c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f28037fe700 1 -- 172.21.15.79:0/3111987277 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000b8a0 con 0x7f280406d8a0 2023-06-08T03:50:52.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f28037fe700 1 -- 172.21.15.79:0/3111987277 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f000ba00 con 0x7f280406d8a0 2023-06-08T03:50:52.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.639+0000 7f28037fe700 1 -- 172.21.15.79:0/3111987277 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000bd90 con 0x7f280406d8a0 2023-06-08T03:50:52.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 -- 172.21.15.79:0/3111987277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f280406d8a0 msgr2=0x7f2804071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 --2- 172.21.15.79:0/3111987277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f280406d8a0 0x7f2804071fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f27f00098f0 tx=0x7f27f0009c00 comp rx=0 tx=0).stop 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 -- 172.21.15.79:0/3111987277 shutdown_connections 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 --2- 172.21.15.79:0/3111987277 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2804069d50 0x7f280406a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 --2- 172.21.15.79:0/3111987277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f280406d8a0 0x7f2804071fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 -- 172.21.15.79:0/3111987277 >> 172.21.15.79:0/3111987277 conn(0x7f28040b1d30 msgr2=0x7f280406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 -- 172.21.15.79:0/3111987277 shutdown_connections 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.640+0000 7f280b067700 1 -- 172.21.15.79:0/3111987277 wait complete. 2023-06-08T03:50:52.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.641+0000 7f280b067700 1 Processor -- start 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.641+0000 7f280b067700 1 -- start start 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f280b067700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 0x7f28040c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f280b067700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280406d8a0 0x7f2804158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f280b067700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f280415c1f0 0x7f2804158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f280b067700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c71f0 con 0x7f2804069d50 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f280b067700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c74d0 con 0x7f280406d8a0 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f280b067700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c77b0 con 0x7f280415c1f0 2023-06-08T03:50:52.649 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f2808e03700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 0x7f28040c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f2808e03700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 0x7f28040c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35556/0 (socket says 172.21.15.79:35556) 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f2808e03700 1 -- 172.21.15.79:0/2323594117 learned_addr learned my addr 172.21.15.79:0/2323594117 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f2803fff700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280406d8a0 0x7f2804158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.642+0000 7f2809604700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f280415c1f0 0x7f2804158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f2808e03700 1 -- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f280415c1f0 msgr2=0x7f2804158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f2808e03700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f280415c1f0 0x7f2804158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f2808e03700 1 -- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280406d8a0 msgr2=0x7f2804158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f2808e03700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280406d8a0 0x7f2804158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f2808e03700 1 -- 172.21.15.79:0/2323594117 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0009550 con 0x7f2804069d50 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f2808e03700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 0x7f28040c88e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f27f800c450 tx=0x7f27f800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80052d0 con 0x7f2804069d50 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8005430 con 0x7f2804069d50 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040c7a90 con 0x7f2804069d50 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80038c0 con 0x7f2804069d50 2023-06-08T03:50:52.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.643+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28040c8e20 con 0x7f2804069d50 2023-06-08T03:50:52.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.644+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28040c7fb0 con 0x7f2804069d50 2023-06-08T03:50:52.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.644+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27f8003a20 con 0x7f2804069d50 2023-06-08T03:50:52.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.645+0000 7f28017fa700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f27e806ace0 0x7f27e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.645+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f27f8088a40 con 0x7f2804069d50 2023-06-08T03:50:52.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.645+0000 7f2803fff700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f27e806ace0 0x7f27e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.645+0000 7f2803fff700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f27e806ace0 0x7f27e806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f27f0014040 tx=0x7f27f0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.649+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f28040c7fb0 con 0x7f2804069d50 2023-06-08T03:50:52.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.835+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f28040c7fb0 con 0x7f2804069d50 2023-06-08T03:50:52.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.835+0000 7f28017fa700 1 -- 172.21.15.79:0/2323594117 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f28040c7fb0 con 0x7f2804069d50 2023-06-08T03:50:52.839 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:50:52.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f27e806ace0 msgr2=0x7f27e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f27e806ace0 0x7f27e806d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f27f0014040 tx=0x7f27f0008030 comp rx=0 tx=0).stop 2023-06-08T03:50:52.841 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 msgr2=0x7f28040c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 0x7f28040c88e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f27f800c450 tx=0x7f27f800c760 comp rx=0 tx=0).stop 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 shutdown_connections 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f27e806ace0 0x7f27e806d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f280415c1f0 0x7f2804158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2804069d50 0x7f28040c88e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 --2- 172.21.15.79:0/2323594117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f280406d8a0 0x7f2804158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.842 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 >> 172.21.15.79:0/2323594117 conn(0x7f28040b1d30 msgr2=0x7f28040cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:52.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 shutdown_connections 2023-06-08T03:50:52.843 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.837+0000 7f280b067700 1 -- 172.21.15.79:0/2323594117 wait complete. 2023-06-08T03:50:52.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.971+0000 7f30af9ec700 1 Processor -- start 2023-06-08T03:50:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.971+0000 7f30af9ec700 1 -- start start 2023-06-08T03:50:52.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30af9ec700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f30a80b1650 0x7f30a80ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30af9ec700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30a8160080 0x7f30a80cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30af9ec700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a80cea60 con 0x7f30a806b0d0 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30af9ec700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a80cf6e0 con 0x7f30a80b1650 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30af9ec700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a80ced40 con 0x7f30a8160080 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30acf87700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f30a80b1650 0x7f30a80ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30ad788700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f30a806b0d0 0x7f30a815f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59038/0 (socket says 172.21.15.79:59038) 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30ad788700 1 -- 172.21.15.79:0/2769263181 learned_addr learned my addr 172.21.15.79:0/2769263181 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30adf89700 1 --2- 172.21.15.79:0/2769263181 >> v2:172.21.15.156:3300/0 conn(0x7f30a8160080 0x7f30a80cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30acf87700 1 -- 172.21.15.79:0/2769263181 >> v2:172.21.15.156:3300/0 conn(0x7f30a8160080 msgr2=0x7f30a80cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30acf87700 1 --2- 172.21.15.79:0/2769263181 >> v2:172.21.15.156:3300/0 conn(0x7f30a8160080 0x7f30a80cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30acf87700 1 -- 172.21.15.79:0/2769263181 >> v1:172.21.15.79:6789/0 conn(0x7f30a806b0d0 legacy=0x7f30a815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30acf87700 1 -- 172.21.15.79:0/2769263181 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a80cf020 con 0x7f30a80b1650 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f309ffff700 1 -- 172.21.15.79:0/2769263181 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779711836 0 0) 0x7f30a80cea60 con 0x7f30a806b0d0 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f30acf87700 1 --2- 172.21.15.79:0/2769263181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80b1650 0x7f30a80ce460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30a80c7820 tx=0x7f3098009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef38454d71be786b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f309ffff700 1 -- 172.21.15.79:0/2769263181 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098011830 con 0x7f30a80b1650 2023-06-08T03:50:52.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f309ffff700 1 -- 172.21.15.79:0/2769263181 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3098011990 con 0x7f30a80b1650 2023-06-08T03:50:52.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.972+0000 7f309ffff700 1 -- 172.21.15.79:0/2769263181 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098011d20 con 0x7f30a80b1650 2023-06-08T03:50:52.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 -- 172.21.15.79:0/2769263181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80b1650 msgr2=0x7f30a80ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 --2- 172.21.15.79:0/2769263181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80b1650 0x7f30a80ce460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f30a80c7820 tx=0x7f3098009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:52.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 -- 172.21.15.79:0/2769263181 shutdown_connections 2023-06-08T03:50:52.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 --2- 172.21.15.79:0/2769263181 >> v2:172.21.15.156:3300/0 conn(0x7f30a8160080 0x7f30a80cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 --2- 172.21.15.79:0/2769263181 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80b1650 0x7f30a80ce460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 -- 172.21.15.79:0/2769263181 >> 172.21.15.79:0/2769263181 conn(0x7f30a806a670 msgr2=0x7f30a8099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 -- 172.21.15.79:0/2769263181 shutdown_connections 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.973+0000 7f30af9ec700 1 -- 172.21.15.79:0/2769263181 wait complete. 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.974+0000 7f30af9ec700 1 Processor -- start 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.974+0000 7f30af9ec700 1 -- start start 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30af9ec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 0x7f30a81f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30af9ec700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30a80b1650 0x7f30a815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30af9ec700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a8160080 0x7f30a81f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30af9ec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81f2c70 con 0x7f30a806b0d0 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30af9ec700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81f1c60 con 0x7f30a80b1650 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30af9ec700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81f5d40 con 0x7f30a8160080 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 0x7f30a81f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 0x7f30a81f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35568/0 (socket says 172.21.15.79:35568) 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 -- 172.21.15.79:0/3936574532 learned_addr learned my addr 172.21.15.79:0/3936574532 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:52.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30acf87700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30a80b1650 0x7f30a815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30adf89700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a8160080 0x7f30a81f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 -- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a8160080 msgr2=0x7f30a81f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a8160080 0x7f30a81f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 -- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30a80b1650 msgr2=0x7f30a815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30a80b1650 0x7f30a815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 -- 172.21.15.79:0/3936574532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980094a0 con 0x7f30a806b0d0 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30adf89700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a8160080 0x7f30a81f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:52.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.975+0000 7f30ad788700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 0x7f30a81f2730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c550 tx=0x7f30a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.976+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4004c80 con 0x7f30a806b0d0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.976+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a4004e00 con 0x7f30a806b0d0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.976+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a81f6020 con 0x7f30a806b0d0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.976+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4007310 con 0x7f30a806b0d0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.976+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a81f2210 con 0x7f30a806b0d0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.977+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a806c0a0 con 0x7f30a806b0d0 2023-06-08T03:50:52.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.977+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f30a4007470 con 0x7f30a806b0d0 2023-06-08T03:50:52.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.978+0000 7f309dffb700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308c06ace0 0x7f308c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:52.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.978+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f30a4089550 con 0x7f30a806b0d0 2023-06-08T03:50:52.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.978+0000 7f30acf87700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308c06ace0 0x7f308c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:52.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.978+0000 7f30acf87700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308c06ace0 0x7f308c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30980090b0 tx=0x7f3098008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:52.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:52.982+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30a4054270 con 0x7f30a806b0d0 2023-06-08T03:50:53.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.212+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30a8073e90 con 0x7f30a806b0d0 2023-06-08T03:50:53.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.213+0000 7f309dffb700 1 -- 172.21.15.79:0/3936574532 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f30a4057890 con 0x7f30a806b0d0 2023-06-08T03:50:53.217 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:53.217 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:50:53.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308c06ace0 msgr2=0x7f308c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308c06ace0 0x7f308c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30980090b0 tx=0x7f3098008030 comp rx=0 tx=0).stop 2023-06-08T03:50:53.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 msgr2=0x7f30a81f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 0x7f30a81f2730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c550 tx=0x7f30a400c910 comp rx=0 tx=0).stop 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 shutdown_connections 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308c06ace0 0x7f308c06d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a8160080 0x7f30a81f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a806b0d0 0x7f30a81f2730 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 --2- 172.21.15.79:0/3936574532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30a80b1650 0x7f30a815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 >> 172.21.15.79:0/3936574532 conn(0x7f30a806a670 msgr2=0x7f30a80b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:53.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 shutdown_connections 2023-06-08T03:50:53.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.215+0000 7f30af9ec700 1 -- 172.21.15.79:0/3936574532 wait complete. 2023-06-08T03:50:53.221 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:50:53.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:50:53.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.346+0000 7f4cdfe98700 1 Processor -- start 2023-06-08T03:50:53.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.346+0000 7f4cdfe98700 1 -- start start 2023-06-08T03:50:53.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.346+0000 7f4cdfe98700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cdfe98700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4cd80cd6c0 0x7f4cd806a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cdfe98700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd80cee20 con 0x7f4cd806f210 2023-06-08T03:50:53.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cdfe98700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd80cf330 con 0x7f4cd80cd6c0 2023-06-08T03:50:53.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cdfe98700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd806fa00 con 0x7f4cd815f6c0 2023-06-08T03:50:53.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:49898/0 (socket says 172.21.15.79:49898) 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 -- 172.21.15.79:0/4289455721 learned_addr learned my addr 172.21.15.79:0/4289455721 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cdd433700 1 --2- 172.21.15.79:0/4289455721 >> v2:172.21.15.156:3300/0 conn(0x7f4cd80cd6c0 0x7f4cd806a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 -- 172.21.15.79:0/4289455721 >> v2:172.21.15.156:3300/0 conn(0x7f4cd80cd6c0 msgr2=0x7f4cd806a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 --2- 172.21.15.79:0/4289455721 >> v2:172.21.15.156:3300/0 conn(0x7f4cd80cd6c0 0x7f4cd806a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 -- 172.21.15.79:0/4289455721 >> v1:172.21.15.156:6789/0 conn(0x7f4cd806f210 legacy=0x7f4cd80cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.347+0000 7f4cddc34700 1 -- 172.21.15.79:0/4289455721 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd80cd1e0 con 0x7f4cd815f6c0 2023-06-08T03:50:53.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cddc34700 1 --2- 172.21.15.79:0/4289455721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8069d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80098f0 tx=0x7f4cc8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdbd6bd3e6f20acf server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdcc32700 1 -- 172.21.15.79:0/4289455721 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8011830 con 0x7f4cd815f6c0 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdcc32700 1 -- 172.21.15.79:0/4289455721 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8011990 con 0x7f4cd815f6c0 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdcc32700 1 -- 172.21.15.79:0/4289455721 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8011d20 con 0x7f4cd815f6c0 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 -- 172.21.15.79:0/4289455721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 msgr2=0x7f4cd8069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/4289455721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8069d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80098f0 tx=0x7f4cc8009c00 comp rx=0 tx=0).stop 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 -- 172.21.15.79:0/4289455721 shutdown_connections 2023-06-08T03:50:53.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/4289455721 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8069d50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/4289455721 >> v2:172.21.15.156:3300/0 conn(0x7f4cd80cd6c0 0x7f4cd806a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 -- 172.21.15.79:0/4289455721 >> 172.21.15.79:0/4289455721 conn(0x7f4cd80b1d20 msgr2=0x7f4cd806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:53.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 -- 172.21.15.79:0/4289455721 shutdown_connections 2023-06-08T03:50:53.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.348+0000 7f4cdfe98700 1 -- 172.21.15.79:0/4289455721 wait complete. 2023-06-08T03:50:53.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.349+0000 7f4cdfe98700 1 Processor -- start 2023-06-08T03:50:53.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.349+0000 7f4cdfe98700 1 -- start start 2023-06-08T03:50:53.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdfe98700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 0x7f4cd81e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdfe98700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cd80cd6c0 0x7f4cd81e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdfe98700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdfe98700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd815a3e0 con 0x7f4cd806f210 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdfe98700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd81e6c70 con 0x7f4cd80cd6c0 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdfe98700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd81e6f50 con 0x7f4cd815f6c0 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cddc34700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 0x7f4cd81e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cddc34700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 0x7f4cd81e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35580/0 (socket says 172.21.15.79:35580) 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cddc34700 1 -- 172.21.15.79:0/3029590518 learned_addr learned my addr 172.21.15.79:0/3029590518 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:53.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cdd433700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cd80cd6c0 0x7f4cd81e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.350+0000 7f4cde435700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cddc34700 1 -- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 msgr2=0x7f4cd8158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cddc34700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cddc34700 1 -- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cd80cd6c0 msgr2=0x7f4cd81e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cddc34700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cd80cd6c0 0x7f4cd81e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cddc34700 1 -- 172.21.15.79:0/3029590518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8009550 con 0x7f4cd806f210 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cddc34700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 0x7f4cd81e6690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8000c00 tx=0x7f4cc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:53.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8004400 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc801f070 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd81e7230 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80038e0 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.351+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd81e7510 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.352+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd8073e90 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.353+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4cc8021030 con 0x7f4cd806f210 2023-06-08T03:50:53.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.353+0000 7f4cce7fc700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4cbc06ace0 0x7f4cbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.353+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4cc808ab00 con 0x7f4cd806f210 2023-06-08T03:50:53.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.353+0000 7f4cdd433700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4cbc06ace0 0x7f4cbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.354+0000 7f4cdd433700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4cbc06ace0 0x7f4cbc06d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80aff40 tx=0x7f4cd4009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:53.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.357+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4cc80558d0 con 0x7f4cd806f210 2023-06-08T03:50:53.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.546+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cd80ae8f0 con 0x7f4cd806f210 2023-06-08T03:50:53.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.547+0000 7f4cce7fc700 1 -- 172.21.15.79:0/3029590518 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f4cc8058ef0 con 0x7f4cd806f210 2023-06-08T03:50:53.552 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:53.552 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:50:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4cbc06ace0 msgr2=0x7f4cbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4cbc06ace0 0x7f4cbc06d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80aff40 tx=0x7f4cd4009210 comp rx=0 tx=0).stop 2023-06-08T03:50:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 msgr2=0x7f4cd81e6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 0x7f4cd81e6690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8000c00 tx=0x7f4cc8004750 comp rx=0 tx=0).stop 2023-06-08T03:50:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 shutdown_connections 2023-06-08T03:50:53.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4cbc06ace0 0x7f4cbc06d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4cd815f6c0 0x7f4cd8158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4cd806f210 0x7f4cd81e6690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 --2- 172.21.15.79:0/3029590518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4cd80cd6c0 0x7f4cd81e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 >> 172.21.15.79:0/3029590518 conn(0x7f4cd80b1d20 msgr2=0x7f4cd806eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:53.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 shutdown_connections 2023-06-08T03:50:53.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.550+0000 7f4cdfe98700 1 -- 172.21.15.79:0/3029590518 wait complete. 2023-06-08T03:50:53.566 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:50:53.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.681+0000 7f5484f54700 1 Processor -- start 2023-06-08T03:50:53.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.681+0000 7f5484f54700 1 -- start start 2023-06-08T03:50:53.686 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f5484f54700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54800cc900 0x7f548006b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f5484f54700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548006b700 con 0x7f54800ca7c0 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f5484f54700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54800ccd80 con 0x7f54800707e0 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f5484f54700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54800cfc90 con 0x7f54800cc900 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f547dd9b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54800cc900 0x7f548006b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f547e59c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f54800ca7c0 0x7f548006a6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59064/0 (socket says 172.21.15.79:59064) 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f547e59c700 1 -- 172.21.15.79:0/765525992 learned_addr learned my addr 172.21.15.79:0/765525992 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:53.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f547dd9b700 1 -- 172.21.15.79:0/765525992 >> v1:172.21.15.156:6789/0 conn(0x7f54800707e0 legacy=0x7f5480070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f547dd9b700 1 -- 172.21.15.79:0/765525992 >> v1:172.21.15.79:6789/0 conn(0x7f54800ca7c0 legacy=0x7f548006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.682+0000 7f547dd9b700 1 -- 172.21.15.79:0/765525992 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800cf710 con 0x7f54800cc900 2023-06-08T03:50:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f547dd9b700 1 --2- 172.21.15.79:0/765525992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f548006b1c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5470009840 tx=0x7f5470009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a4f8a46eed78d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f547d59a700 1 -- 172.21.15.79:0/765525992 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470004230 con 0x7f54800cc900 2023-06-08T03:50:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f547d59a700 1 -- 172.21.15.79:0/765525992 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470004390 con 0x7f54800cc900 2023-06-08T03:50:53.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f547d59a700 1 -- 172.21.15.79:0/765525992 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700036a0 con 0x7f54800cc900 2023-06-08T03:50:53.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 -- 172.21.15.79:0/765525992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 msgr2=0x7f548006b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 --2- 172.21.15.79:0/765525992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f548006b1c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5470009840 tx=0x7f5470009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 -- 172.21.15.79:0/765525992 shutdown_connections 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 --2- 172.21.15.79:0/765525992 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f548006b1c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 -- 172.21.15.79:0/765525992 >> 172.21.15.79:0/765525992 conn(0x7f54800b14a0 msgr2=0x7f54800cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 -- 172.21.15.79:0/765525992 shutdown_connections 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.683+0000 7f5484f54700 1 -- 172.21.15.79:0/765525992 wait complete. 2023-06-08T03:50:53.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 Processor -- start 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 -- start start 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54800707e0 0x7f548014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800ca7c0 0x7f54800c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f54800c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b3530 con 0x7f54800cc900 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b3710 con 0x7f54800ca7c0 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.685+0000 7f5484f54700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54800b39f0 con 0x7f54800707e0 2023-06-08T03:50:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f54800c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f54800c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35598/0 (socket says 172.21.15.79:35598) 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 -- 172.21.15.79:0/167991441 learned_addr learned my addr 172.21.15.79:0/167991441 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547e59c700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54800707e0 0x7f548014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547dd9b700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800ca7c0 0x7f54800c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 -- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54800707e0 msgr2=0x7f548014d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54800707e0 0x7f548014d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 -- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800ca7c0 msgr2=0x7f54800c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800ca7c0 0x7f54800c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 -- 172.21.15.79:0/167991441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700094a0 con 0x7f54800cc900 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547dd9b700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800ca7c0 0x7f54800c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547e59c700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54800707e0 0x7f548014d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f547ed9d700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f54800c84a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f547800b990 tx=0x7f547800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547800c9d0 con 0x7f54800cc900 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547800cb30 con 0x7f54800cc900 2023-06-08T03:50:53.694 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54800b3cd0 con 0x7f54800cc900 2023-06-08T03:50:53.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5478014440 con 0x7f54800cc900 2023-06-08T03:50:53.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.686+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54800b4290 con 0x7f54800cc900 2023-06-08T03:50:53.695 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.687+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480073e90 con 0x7f54800cc900 2023-06-08T03:50:53.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.688+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5478006d50 con 0x7f54800cc900 2023-06-08T03:50:53.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.688+0000 7f546effd700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f546006ace0 0x7f546006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:53.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.688+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5478089260 con 0x7f54800cc900 2023-06-08T03:50:53.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.688+0000 7f547e59c700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f546006ace0 0x7f546006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:53.696 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.689+0000 7f547e59c700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f546006ace0 0x7f546006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54800c8d30 tx=0x7f5474009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:53.697 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.692+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5478054020 con 0x7f54800cc900 2023-06-08T03:50:53.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.918+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54800713e0 con 0x7f54800cc900 2023-06-08T03:50:53.922 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.918+0000 7f546effd700 1 -- 172.21.15.79:0/167991441 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f5478057640 con 0x7f54800cc900 2023-06-08T03:50:53.923 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:50:53.923 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f546006ace0 msgr2=0x7f546006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f546006ace0 0x7f546006d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54800c8d30 tx=0x7f5474009040 comp rx=0 tx=0).stop 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 msgr2=0x7f54800c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f54800c84a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f547800b990 tx=0x7f547800bd50 comp rx=0 tx=0).stop 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 shutdown_connections 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54800cc900 0x7f54800c84a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.925 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f546006ace0 0x7f546006d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54800707e0 0x7f548014d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.920+0000 7f5484f54700 1 --2- 172.21.15.79:0/167991441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54800ca7c0 0x7f54800c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:53.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.921+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 >> 172.21.15.79:0/167991441 conn(0x7f54800b14a0 msgr2=0x7f548015e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:53.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.921+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 shutdown_connections 2023-06-08T03:50:53.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:53.921+0000 7f5484f54700 1 -- 172.21.15.79:0/167991441 wait complete. 2023-06-08T03:50:53.926 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:50:54.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-08T03:50:24.668853+0000', 'modified': '2023-06-08T03:50:41.945158+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6833/2931327716', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6832', 'nonce': 2931327716}, {'type': 'v1', 'addr': '172.21.15.79:6833', 'nonce': 2931327716}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-08T03:50:54.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-08T03:50:54.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-08T03:50:54.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.456+0000 7f4594425700 1 Processor -- start 2023-06-08T03:50:54.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.457+0000 7f4594425700 1 -- start start 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.457+0000 7f4594425700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0d0060 0x7f458c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.457+0000 7f4594425700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f458c0cd080 0x7f458c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.457+0000 7f4594425700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0cae50 con 0x7f458c0b20f0 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.457+0000 7f4594425700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c0cbea0 con 0x7f458c0cd080 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.457+0000 7f4594425700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0cda30 con 0x7f458c0d0060 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45919c0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f458c0cd080 0x7f458c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45929c2700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f458c0b20f0 0x7f458c0b2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59082/0 (socket says 172.21.15.79:59082) 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45929c2700 1 -- 172.21.15.79:0/768981351 learned_addr learned my addr 172.21.15.79:0/768981351 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45921c1700 1 --2- 172.21.15.79:0/768981351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0d0060 0x7f458c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45919c0700 1 -- 172.21.15.79:0/768981351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0d0060 msgr2=0x7f458c0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45919c0700 1 --2- 172.21.15.79:0/768981351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0d0060 0x7f458c0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45919c0700 1 -- 172.21.15.79:0/768981351 >> v1:172.21.15.79:6789/0 conn(0x7f458c0b20f0 legacy=0x7f458c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45919c0700 1 -- 172.21.15.79:0/768981351 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c15d930 con 0x7f458c0cd080 2023-06-08T03:50:54.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45911bf700 1 -- 172.21.15.79:0/768981351 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103156321 0 0) 0x7f458c0cae50 con 0x7f458c0b20f0 2023-06-08T03:50:54.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45919c0700 1 --2- 172.21.15.79:0/768981351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0cd080 0x7f458c0cd4f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4588009840 tx=0x7f4588009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b2c95438b83d10 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45911bf700 1 -- 172.21.15.79:0/768981351 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588011830 con 0x7f458c0cd080 2023-06-08T03:50:54.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45911bf700 1 -- 172.21.15.79:0/768981351 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4588011990 con 0x7f458c0cd080 2023-06-08T03:50:54.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.458+0000 7f45911bf700 1 -- 172.21.15.79:0/768981351 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588011d20 con 0x7f458c0cd080 2023-06-08T03:50:54.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 -- 172.21.15.79:0/768981351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0cd080 msgr2=0x7f458c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 --2- 172.21.15.79:0/768981351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0cd080 0x7f458c0cd4f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4588009840 tx=0x7f4588009b50 comp rx=0 tx=0).stop 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 -- 172.21.15.79:0/768981351 shutdown_connections 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 --2- 172.21.15.79:0/768981351 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0d0060 0x7f458c0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 --2- 172.21.15.79:0/768981351 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0cd080 0x7f458c0cd4f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 -- 172.21.15.79:0/768981351 >> 172.21.15.79:0/768981351 conn(0x7f458c06e1e0 msgr2=0x7f458c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 -- 172.21.15.79:0/768981351 shutdown_connections 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.459+0000 7f4594425700 1 -- 172.21.15.79:0/768981351 wait complete. 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.460+0000 7f4594425700 1 Processor -- start 2023-06-08T03:50:54.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.460+0000 7f4594425700 1 -- start start 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4594425700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0b20f0 0x7f458c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4594425700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458c0cd080 0x7f458c0c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4594425700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 0x7f458c1e4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4594425700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1e5e70 con 0x7f458c0d0060 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4594425700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0c4af0 con 0x7f458c0cd080 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4594425700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1e46f0 con 0x7f458c0b20f0 2023-06-08T03:50:54.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 0x7f458c1e4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 0x7f458c1e4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35612/0 (socket says 172.21.15.79:35612) 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 -- 172.21.15.79:0/3712401409 learned_addr learned my addr 172.21.15.79:0/3712401409 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45921c1700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0b20f0 0x7f458c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45919c0700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458c0cd080 0x7f458c0c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 -- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0b20f0 msgr2=0x7f458c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0b20f0 0x7f458c0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 -- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458c0cd080 msgr2=0x7f458c0c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458c0cd080 0x7f458c0c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 -- 172.21.15.79:0/3712401409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880094a0 con 0x7f458c0d0060 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45919c0700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458c0cd080 0x7f458c0c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f45929c2700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 0x7f458c1e4930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f458400cb90 tx=0x7f458400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4582ffd700 1 -- 172.21.15.79:0/3712401409 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458400aa80 con 0x7f458c0d0060 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.461+0000 7f4582ffd700 1 -- 172.21.15.79:0/3712401409 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458400abe0 con 0x7f458c0d0060 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.462+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c1e4410 con 0x7f458c0d0060 2023-06-08T03:50:54.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.462+0000 7f4582ffd700 1 -- 172.21.15.79:0/3712401409 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840070d0 con 0x7f458c0d0060 2023-06-08T03:50:54.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.462+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c0c4780 con 0x7f458c0d0060 2023-06-08T03:50:54.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.463+0000 7f4582ffd700 1 -- 172.21.15.79:0/3712401409 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4584007230 con 0x7f458c0d0060 2023-06-08T03:50:54.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.463+0000 7f4582ffd700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f457006ace0 0x7f457006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.463+0000 7f4582ffd700 1 -- 172.21.15.79:0/3712401409 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f45840895a0 con 0x7f458c0d0060 2023-06-08T03:50:54.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.463+0000 7f45921c1700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f457006ace0 0x7f457006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.464+0000 7f45921c1700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f457006ace0 0x7f457006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f458c1531e0 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.465+0000 7f45a05aab80 1 Processor -- start 2023-06-08T03:50:54.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.465+0000 7f45a05aab80 1 -- start start 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3e74a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3e74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45a05aab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3d3d3b0 con 0x562ce3e7ebc0 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3d3ec60 con 0x562ce3e36a50 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ce3d91570 con 0x562ce3e60c10 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35620/0 (socket says 172.21.15.79:35620) 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 -- 172.21.15.79:0/2960542383 learned_addr learned my addr 172.21.15.79:0/2960542383 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45919c0700 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3e74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45921c1700 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3e74a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 -- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 msgr2=0x562ce3e74a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3e74a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 -- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 msgr2=0x562ce3e74f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3e74f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.466+0000 7f45929c2700 1 -- 172.21.15.79:0/2960542383 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce3bb8d90 con 0x562ce3e7ebc0 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45929c2700 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e634b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4584057bf0 tx=0x7f458405bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1d195fa466eb3f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f456ffff700 1 -- 172.21.15.79:0/2960542383 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584056320 con 0x562ce3e7ebc0 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f456ffff700 1 -- 172.21.15.79:0/2960542383 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458405fe40 con 0x562ce3e7ebc0 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f456ffff700 1 -- 172.21.15.79:0/2960542383 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458406bb80 con 0x562ce3e7ebc0 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 -- 172.21.15.79:0/2960542383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 msgr2=0x562ce3e634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e634b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4584057bf0 tx=0x7f458405bf40 comp rx=0 tx=0).stop 2023-06-08T03:50:54.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 -- 172.21.15.79:0/2960542383 shutdown_connections 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e634b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3e74a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2960542383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3e74f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.467+0000 7f45a05aab80 1 -- 172.21.15.79:0/2960542383 >> 172.21.15.79:0/2960542383 conn(0x562ce3e35ce0 msgr2=0x562ce3e7e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.468+0000 7f45a05aab80 1 -- 172.21.15.79:0/2960542383 shutdown_connections 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.468+0000 7f45a05aab80 1 -- 172.21.15.79:0/2960542383 wait complete. 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.468+0000 7f45a05aab80 1 -- start start 2023-06-08T03:50:54.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.468+0000 7f45a05aab80 1 Processor -- start 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e36a50 0x562ce3e5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e60c10 0x562ce3e66b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3d8aa00 con 0x562ce3e7ebc0 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3d2aa70 con 0x562ce3e60c10 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ce3d2a770 con 0x562ce3e36a50 2023-06-08T03:50:54.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45a05aab80 10 client.0 _ll_register_callbacks cb 0x562ce3d28630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e67050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35626/0 (socket says 172.21.15.79:35626) 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 -- 172.21.15.79:0/1813950735 learned_addr learned my addr 172.21.15.79:0/1813950735 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45919c0700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e60c10 0x562ce3e66b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45921c1700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e36a50 0x562ce3e5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 -- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e36a50 msgr2=0x562ce3e5f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e36a50 0x562ce3e5f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 -- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e60c10 msgr2=0x562ce3e66b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e60c10 0x562ce3e66b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.469+0000 7f45929c2700 1 -- 172.21.15.79:0/1813950735 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c0c4780 con 0x562ce3e7ebc0 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45929c2700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e67050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f458400a340 tx=0x7f458400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665170423ffdea46 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f456d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f456d7fa700 1 -- 172.21.15.79:0/1813950735 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840872e0 con 0x562ce3e7ebc0 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f456d7fa700 1 -- 172.21.15.79:0/1813950735 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4584089070 con 0x562ce3e7ebc0 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ce3e2b850 con 0x562ce3e7ebc0 2023-06-08T03:50:54.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f456d7fa700 1 -- 172.21.15.79:0/1813950735 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4584089230 con 0x562ce3e7ebc0 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 10 client.4851 fetch_fsmap learned FSMap version 9 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562ce3ba1c10 con 0x562ce3e7ebc0 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f456d7fa700 1 -- 172.21.15.79:0/1813950735 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458407a030 con 0x562ce3e7ebc0 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f456d7fa700 1 -- 172.21.15.79:0/1813950735 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f4584053d70 con 0x562ce3e7ebc0 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 10 client.4851 fetch_fsmap finished waiting for FSMap version 9 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 10 client.4851 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-08T03:50:54.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 20 client.4851 populate_metadata read hostname 'smithi079' 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3e69400 0x562ce3e697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 4 client.4851 mds_command: new command op to 4260 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 --> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ce3d839e0 con 0x562ce3e69400 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.470+0000 7f45921c1700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3e69400 0x562ce3e697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f456d7fa700 10 client.4851 ms_handle_connect on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45921c1700 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3e69400 0x562ce3e697d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ac2b1a29d61cb7 server_cookie=a1989af7cd6e73e in_seq=0 out_seq=0 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f456d7fa700 1 -- 172.21.15.79:0/1813950735 <== mds.0 v2:172.21.15.79:6832/2931327716 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ce3d839e0 con 0x562ce3e69400 2023-06-08T03:50:54.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f456d7fa700 10 client.4851 handle_command_reply: tid=0 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 client.4851 shutdown 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 msgr2=0x562ce3e67050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e67050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f458400a340 tx=0x7f458400af90 comp rx=0 tx=0).stop 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 shutdown_connections 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e7ebc0 0x562ce3e67050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3e69400 0x562ce3e697d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e36a50 0x562ce3e5f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f456d7fa700 0 client.4851 ms_handle_reset on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 --2- 172.21.15.79:0/1813950735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e60c10 0x562ce3e66b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 >> 172.21.15.79:0/1813950735 conn(0x562ce3e35ce0 msgr2=0x562ce3e34550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 shutdown_connections 2023-06-08T03:50:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.471+0000 7f45a05aab80 1 -- 172.21.15.79:0/1813950735 wait complete. 2023-06-08T03:50:54.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.472+0000 7f45a05aab80 20 client.4851 trim_cache size 0 max 16384 2023-06-08T03:50:54.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.489+0000 7f45a05aab80 1 Processor -- start 2023-06-08T03:50:54.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.489+0000 7f45a05aab80 1 -- start start 2023-06-08T03:50:54.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 0x562ce3e349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3ea0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.494 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e7ebc0 0x562ce3e39520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45a05aab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3c0e8d0 con 0x562ce3e36a50 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3e5f020 con 0x562ce3e7ebc0 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ce3e6b940 con 0x562ce3e60c10 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 0x562ce3e349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 0x562ce3e349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35630/0 (socket says 172.21.15.79:35630) 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 -- 172.21.15.79:0/2119819401 learned_addr learned my addr 172.21.15.79:0/2119819401 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:54.495 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45919c0700 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3ea0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45929c2700 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e7ebc0 0x562ce3e39520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 -- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 msgr2=0x562ce3ea0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3ea0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 -- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e7ebc0 msgr2=0x562ce3e39520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e7ebc0 0x562ce3e39520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 -- 172.21.15.79:0/2119819401 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ce3bdf200 con 0x562ce3e36a50 2023-06-08T03:50:54.496 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45929c2700 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e7ebc0 0x562ce3e39520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:50:54.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f45921c1700 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 0x562ce3e349d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x562ce3e61760 tx=0x7f457c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccc3a29b4cd61de server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f4580dac700 1 -- 172.21.15.79:0/2119819401 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c02b520 con 0x562ce3e36a50 2023-06-08T03:50:54.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f4580dac700 1 -- 172.21.15.79:0/2119819401 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c02b8d0 con 0x562ce3e36a50 2023-06-08T03:50:54.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.490+0000 7f4580dac700 1 -- 172.21.15.79:0/2119819401 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c02bc40 con 0x562ce3e36a50 2023-06-08T03:50:54.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 -- 172.21.15.79:0/2119819401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 msgr2=0x562ce3e349d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 0x562ce3e349d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x562ce3e61760 tx=0x7f457c017000 comp rx=0 tx=0).stop 2023-06-08T03:50:54.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 -- 172.21.15.79:0/2119819401 shutdown_connections 2023-06-08T03:50:54.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e7ebc0 0x562ce3e39520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e36a50 0x562ce3e349d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 --2- 172.21.15.79:0/2119819401 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e60c10 0x562ce3ea0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 -- 172.21.15.79:0/2119819401 >> 172.21.15.79:0/2119819401 conn(0x562ce3e35ce0 msgr2=0x562ce3e33040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 -- 172.21.15.79:0/2119819401 shutdown_connections 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 -- 172.21.15.79:0/2119819401 wait complete. 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.491+0000 7f45a05aab80 1 -- start start 2023-06-08T03:50:54.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 Processor -- start 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3ea0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 0x562ce3ee57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e7ebc0 0x562ce3ee6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3e7ac50 con 0x562ce3e60c10 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ce3ee3a00 con 0x562ce3e36a50 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ce3e7e220 con 0x562ce3e7ebc0 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.492+0000 7f45a05aab80 10 client.0 _ll_register_callbacks cb 0x562ce3d28630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-08T03:50:54.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 0x562ce3ee57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 0x562ce3ee57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35638/0 (socket says 172.21.15.79:35638) 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 -- 172.21.15.79:0/3489119814 learned_addr learned my addr 172.21.15.79:0/3489119814 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45929c2700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e7ebc0 0x562ce3ee6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45921c1700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3ea0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 -- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e7ebc0 msgr2=0x562ce3ee6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e7ebc0 0x562ce3ee6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 -- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 msgr2=0x562ce3ea0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3ea0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 -- 172.21.15.79:0/3489119814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c00e040 con 0x562ce3e60c10 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45919c0700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 0x562ce3ee57c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4588013000 tx=0x7f45880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60693d98b8f282b8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f456d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f456d7fa700 1 -- 172.21.15.79:0/3489119814 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588004450 con 0x562ce3e60c10 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f456d7fa700 1 -- 172.21.15.79:0/3489119814 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4588021050 con 0x562ce3e60c10 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f458405fe40 con 0x562ce3e60c10 2023-06-08T03:50:54.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f456d7fa700 1 -- 172.21.15.79:0/3489119814 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45880045b0 con 0x562ce3e60c10 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45a05aab80 10 client.4857 fetch_fsmap learned FSMap version 9 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x562ce3bb7260 con 0x562ce3e60c10 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.493+0000 7f456d7fa700 1 -- 172.21.15.79:0/3489119814 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458405fe40 con 0x562ce3e60c10 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f456d7fa700 1 -- 172.21.15.79:0/3489119814 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f45880036a0 con 0x562ce3e60c10 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 10 client.4857 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 20 client.4857 populate_metadata read hostname 'smithi079' 2023-06-08T03:50:54.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3ee3300 0x562ce3ee6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 4 client.4857 mds_command: new command op to 4260 tid=0[{"prefix": "status"}] 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 --> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ce3d46000 con 0x562ce3ee3300 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45921c1700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3ee3300 0x562ce3ee6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f456d7fa700 10 client.4857 ms_handle_connect on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45921c1700 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3ee3300 0x562ce3ee6670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab9611bd64825a1 server_cookie=ac999b94ff06f64e in_seq=0 out_seq=0 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f456d7fa700 1 -- 172.21.15.79:0/3489119814 <== mds.0 v2:172.21.15.79:6832/2931327716 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562ce3d46000 con 0x562ce3ee3300 2023-06-08T03:50:54.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f456d7fa700 10 client.4857 handle_command_reply: tid=0 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 client.4857 shutdown 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 msgr2=0x562ce3ee57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 0x562ce3ee57c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4588013000 tx=0x7f45880047c0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 shutdown_connections 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:6832/2931327716,v1:172.21.15.79:6833/2931327716] conn(0x562ce3ee3300 0x562ce3ee6670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x562ce3e7ebc0 0x562ce3ee6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562ce3e36a50 0x562ce3ea0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f456d7fa700 0 client.4857 ms_handle_reset on v2:172.21.15.79:6832/2931327716 2023-06-08T03:50:54.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 --2- 172.21.15.79:0/3489119814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562ce3e60c10 0x562ce3ee57c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 >> 172.21.15.79:0/3489119814 conn(0x562ce3e35ce0 msgr2=0x562ce3e630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.494+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 shutdown_connections 2023-06-08T03:50:54.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f45a05aab80 1 -- 172.21.15.79:0/3489119814 wait complete. 2023-06-08T03:50:54.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f45a05aab80 20 client.4857 trim_cache size 0 max 16384 2023-06-08T03:50:54.508 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "141881e0-5474-41d9-8205-1a44c31c5e0b", 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4260, 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2023-06-08T03:50:54.509 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 29.825420239, 2023-06-08T03:50:54.510 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 9, 2023-06-08T03:50:54.510 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 25, 2023-06-08T03:50:54.510 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 25, 2023-06-08T03:50:54.510 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 35.213986474000002 2023-06-08T03:50:54.510 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:50:54.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f457006ace0 msgr2=0x7f457006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f457006ace0 0x7f457006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f458c1531e0 tx=0x7f457c008040 comp rx=0 tx=0).stop 2023-06-08T03:50:54.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 msgr2=0x7f458c1e4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 0x7f458c1e4930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f458400cb90 tx=0x7f458400cea0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 shutdown_connections 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f458c0d0060 0x7f458c1e4930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f457006ace0 0x7f457006d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f458c0b20f0 0x7f458c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 --2- 172.21.15.79:0/3712401409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f458c0cd080 0x7f458c0c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 >> 172.21.15.79:0/3712401409 conn(0x7f458c06e1e0 msgr2=0x7f458c15c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 shutdown_connections 2023-06-08T03:50:54.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.495+0000 7f4594425700 1 -- 172.21.15.79:0/3712401409 wait complete. 2023-06-08T03:50:54.513 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-06-08T03:50:54.523 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-06-08T03:50:54.523 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-06-08T03:50:54.524 INFO:tasks.ceph_fuse:client.0 config is {} 2023-06-08T03:50:54.524 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-08T03:50:54.524 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-08T03:50:54.524 INFO:tasks.ceph_fuse:client.2 config is {} 2023-06-08T03:50:54.525 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-08T03:50:54.525 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-08T03:50:54.525 INFO:tasks.ceph_fuse:client.1 config is {} 2023-06-08T03:50:54.525 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-08T03:50:54.525 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-08T03:50:54.526 INFO:tasks.ceph_fuse:client.3 config is {} 2023-06-08T03:50:54.526 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-08T03:50:54.526 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-08T03:50:54.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:54.527 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-06-08T03:50:54.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:54.548 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-06-08T03:50:54.620 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-06-08T03:50:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T03:50:54.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:54.622 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-08T03:50:54.638 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-06-08T03:50:54.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:54.639 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-06-08T03:50:54.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:54.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2023-06-08T03:50:54.817 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-06-08T03:50:54.817 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-08T03:50:54.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.934+0000 7f949ab61700 1 Processor -- start 2023-06-08T03:50:54.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.934+0000 7f949ab61700 1 -- start start 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f949ab61700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 0x7f94940ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f949ab61700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9494159a10 con 0x7f94940c9630 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f949ab61700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94940cc810 con 0x7f94940cd1e0 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f949ab61700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494159450 con 0x7f94940cd9b0 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94988fd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f94940c9630 0x7f9494158a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59120/0 (socket says 172.21.15.79:59120) 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94988fd700 1 -- 172.21.15.79:0/2124563125 learned_addr learned my addr 172.21.15.79:0/2124563125 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:54.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281317273 0 0) 0x7f9494159a10 con 0x7f94940c9630 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94840035b0 con 0x7f94940c9630 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f9493fff700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 0x7f94940ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1989823241 0 0) 0x7f94840035b0 con 0x7f94940c9630 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 msgr2=0x7f94940ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 0x7f94940ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 >> v1:172.21.15.156:6789/0 conn(0x7f94940cd1e0 legacy=0x7f9494159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494159730 con 0x7f94940c9630 2023-06-08T03:50:54.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f9488001670 con 0x7f94940c9630 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 >> v1:172.21.15.79:6789/0 conn(0x7f94940c9630 legacy=0x7f9494158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9484003aa0 0x7f9484003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f9484004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 0x7f9484005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.935+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494159450 con 0x7f9484004850 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948400bbd0 con 0x7f9484003aa0 2023-06-08T03:50:54.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948400bd90 con 0x7f94940cd9b0 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f9484004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94990fe700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 0x7f9484005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94988fd700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9484003aa0 0x7f9484003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 -- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 msgr2=0x7f9484005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94940cd9b0 0x7f9484005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 -- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9484003aa0 msgr2=0x7f9484003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9484003aa0 0x7f9484003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 -- 172.21.15.79:0/2124563125 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840035b0 con 0x7f9484004850 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94988fd700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9484003aa0 0x7f9484003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f9493fff700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f9484004cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9480009b60 tx=0x7f9484005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8ec2535ab69c2f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000e6c0 con 0x7f9484004850 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948000e820 con 0x7f9484004850 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.936+0000 7f94937fe700 1 -- 172.21.15.79:0/2124563125 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000eb90 con 0x7f9484004850 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f949ab61700 1 -- 172.21.15.79:0/2124563125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 msgr2=0x7f9484004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f949ab61700 1 --2- 172.21.15.79:0/2124563125 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f9484004cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9480009b60 tx=0x7f9484005880 comp rx=0 tx=0).stop 2023-06-08T03:50:54.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f94990fe700 1 -- 172.21.15.79:0/2124563125 reap_dead start 2023-06-08T03:50:54.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f949ab61700 1 -- 172.21.15.79:0/2124563125 shutdown_connections 2023-06-08T03:50:54.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f949ab61700 1 -- 172.21.15.79:0/2124563125 >> 172.21.15.79:0/2124563125 conn(0x7f9494069d50 msgr2=0x7f949406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:54.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f949ab61700 1 -- 172.21.15.79:0/2124563125 shutdown_connections 2023-06-08T03:50:54.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.937+0000 7f949ab61700 1 -- 172.21.15.79:0/2124563125 wait complete. 2023-06-08T03:50:54.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.938+0000 7f949ab61700 1 Processor -- start 2023-06-08T03:50:54.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.938+0000 7f949ab61700 1 -- start start 2023-06-08T03:50:54.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f949ab61700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f94940b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f949ab61700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94940c6110 0x7f9494154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f949ab61700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f949414c230 0x7f9494153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f949ab61700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940b1aa0 con 0x7f9484004850 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f949ab61700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940b4190 con 0x7f94940c6110 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f949ab61700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94941554f0 con 0x7f949414c230 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f94940b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f94940b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35656/0 (socket says 172.21.15.79:35656) 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 -- 172.21.15.79:0/2170756242 learned_addr learned my addr 172.21.15.79:0/2170756242 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:54.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f9493fff700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94940c6110 0x7f9494154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94990fe700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f949414c230 0x7f9494153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 -- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f949414c230 msgr2=0x7f9494153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f949414c230 0x7f9494153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 -- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94940c6110 msgr2=0x7f9494154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94940c6110 0x7f9494154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 -- 172.21.15.79:0/2170756242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940b1450 con 0x7f9484004850 2023-06-08T03:50:54.950 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94988fd700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f94940b36a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94880109c0 tx=0x7f9488010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948800c9e0 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948800cb40 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.939+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488010e60 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.940+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94940b1730 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.940+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949414c6d0 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.941+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494073e90 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.941+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9488012450 con 0x7f9484004850 2023-06-08T03:50:54.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.941+0000 7f94917fa700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f947806ad10 0x7f947806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:54.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.941+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f948808b5c0 con 0x7f9484004850 2023-06-08T03:50:54.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.941+0000 7f9493fff700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f947806ad10 0x7f947806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:54.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.942+0000 7f9493fff700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f947806ad10 0x7f947806d1c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94800090b0 tx=0x7f948001f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:54.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:54.945+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94880564c0 con 0x7f9484004850 2023-06-08T03:50:55.180 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.176+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94940c5390 con 0x7f9484004850 2023-06-08T03:50:55.181 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.177+0000 7f94917fa700 1 -- 172.21.15.79:0/2170756242 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9488059ae0 con 0x7f9484004850 2023-06-08T03:50:55.181 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-08T03:50:55.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f947806ad10 msgr2=0x7f947806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:55.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f947806ad10 0x7f947806d1c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94800090b0 tx=0x7f948001f040 comp rx=0 tx=0).stop 2023-06-08T03:50:55.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 msgr2=0x7f94940b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:55.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f94940b36a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f94880109c0 tx=0x7f9488010cd0 comp rx=0 tx=0).stop 2023-06-08T03:50:55.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 shutdown_connections 2023-06-08T03:50:55.183 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f947806ad10 0x7f947806d1c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:55.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f949414c230 0x7f9494153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:55.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9484004850 0x7f94940b36a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:55.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 --2- 172.21.15.79:0/2170756242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94940c6110 0x7f9494154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:55.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 >> 172.21.15.79:0/2170756242 conn(0x7f9494069d50 msgr2=0x7f94940ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:55.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 shutdown_connections 2023-06-08T03:50:55.184 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:55.179+0000 7f949ab61700 1 -- 172.21.15.79:0/2170756242 wait complete. 2023-06-08T03:50:55.195 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-08T03:50:55.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-08T03:50:55.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-08T03:50:55.196 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-08T03:50:55.196 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-08T03:50:55.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-08T03:50:55.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-08T03:50:55.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-08T03:50:55.197 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-08T03:50:55.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.197 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-08T03:50:55.213 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:55.214 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-08T03:50:55.215 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42332sec preferred_lft 42332sec 2023-06-08T03:50:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::6d1d:ad0f:719a:51d1/64 scope link noprefixroute 2023-06-08T03:50:55.216 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:55.216 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-08T03:50:55.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.216 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:55.217 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:55.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2023-06-08T03:50:55.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2023-06-08T03:50:55.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2023-06-08T03:50:55.218 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-08T03:50:55.218 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:55.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.329 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-08T03:50:55.354 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-08T03:50:55.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.356 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-08T03:50:55.414 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-08T03:50:55.414 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-08T03:50:55.414 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-08T03:50:55.414 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-08T03:50:55.414 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-08T03:50:55.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.416 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:55.416 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:55.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-08T03:50:55.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-08T03:50:55.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-08T03:50:55.417 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:55.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.520 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-08T03:50:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.537 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2023-06-08T03:50:55.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.593 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:55.593 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:55.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-08T03:50:55.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-06-08T03:50:55.594 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:55.685 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-06-08T03:50:55.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.685 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:55.686 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:55.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-06-08T03:50:55.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-08T03:50:55.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-06-08T03:50:55.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-06-08T03:50:55.687 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-06-08T03:50:55.687 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:55.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:55.920 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:55.920 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:55.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2023-06-08T03:50:55.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2023-06-08T03:50:55.921 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:56.028 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-08T03:50:56.029 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:50:56.029 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-08T03:50:56.052 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-08T03:50:56.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:50:56.053 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-08T03:50:56.108 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:50:56.174 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:50:56.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:50:56.240 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:50:56.240 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:50:56.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:56.241 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:50:56.298 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-08T03:50:56.299 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-08T03:50:56.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:50:56.365 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:50:56.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:50:56.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:2023-06-08T03:50:56.424+0000 7f91e07e85c0 -1 init, newargv = 0x561432e38530 newargc=15 2023-06-08T03:50:56.427 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[111386]: starting ceph client 2023-06-08T03:50:56.431 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[111386]: starting fuse 2023-06-08T03:50:56.433 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:50:56.434 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:50:56.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:56.435 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:50:56.491 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-08T03:50:56.491 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-06-08T03:50:56.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-08T03:50:56.492 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-08T03:50:56.492 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-08T03:50:56.492 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-08T03:50:56.493 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-08T03:50:56.493 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:56.493 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-08T03:50:56.493 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-08T03:50:56.493 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-08T03:50:56.494 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-08T03:50:56.494 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:56.494 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-08T03:50:56.494 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-08T03:50:56.495 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-08T03:50:56.495 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-08T03:50:56.495 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:56.496 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-08T03:50:56.496 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-08T03:50:56.497 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-08T03:50:56.497 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-08T03:50:56.497 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-08T03:50:56.498 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-08T03:50:56.498 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-08T03:50:56.498 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-08T03:50:56.499 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:56.499 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.0")) 2023-06-08T03:50:56.499 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-08T03:50:56.612 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.0.111386.asok 2023-06-08T03:50:56.795 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.111386.asok 2023-06-08T03:50:56.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:56.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.111386.asok status 2023-06-08T03:50:56.950 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:50:56.951 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-08T03:50:56.951 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "a02638f808b8fe11bccdd01222c0e6e2eb71120b", 2023-06-08T03:50:56.951 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)", 2023-06-08T03:50:56.951 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "0", 2023-06-08T03:50:56.951 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "111386", 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-08T03:50:56.952 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4872, 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 4872 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:50:56.953 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1711255852 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 1711255852 2023-06-08T03:50:56.954 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.4872 192.168.0.1:0/1711255852", 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.1:0/1711255852", 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 9, 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 25, 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-08T03:50:56.955 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-08T03:50:56.956 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:50:56.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-08T03:50:57.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.078+0000 7ff2161a8700 1 Processor -- start 2023-06-08T03:50:57.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.078+0000 7ff2161a8700 1 -- start start 2023-06-08T03:50:57.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff2161a8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff21006a460 0x7ff2100cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff2161a8700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21006eed0 con 0x7ff2100715e0 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff2161a8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21006fa70 con 0x7ff2100cb2e0 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff2161a8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2100cd6d0 con 0x7ff21006a460 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff21006a460 0x7ff2100cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff21006a460 0x7ff2100cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35682/0 (socket says 172.21.15.79:35682) 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 -- 172.21.15.79:0/4148958241 learned_addr learned my addr 172.21.15.79:0/4148958241 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:57.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20effd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff2100715e0 0x7ff210069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59156/0 (socket says 172.21.15.79:59156) 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 -- 172.21.15.79:0/4148958241 >> v1:172.21.15.156:6789/0 conn(0x7ff2100cb2e0 legacy=0x7ff21006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 -- 172.21.15.79:0/4148958241 >> v1:172.21.15.79:6789/0 conn(0x7ff2100715e0 legacy=0x7ff210069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 -- 172.21.15.79:0/4148958241 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21006eb80 con 0x7ff21006a460 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20e7fc700 1 -- 172.21.15.79:0/4148958241 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574141640 0 0) 0x7ff21006eed0 con 0x7ff2100715e0 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.079+0000 7ff20ffff700 1 --2- 172.21.15.79:0/4148958241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100cd190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff208009e50 tx=0x7ff20800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91da366ad31bbd9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff20e7fc700 1 -- 172.21.15.79:0/4148958241 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800e070 con 0x7ff21006a460 2023-06-08T03:50:57.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff20e7fc700 1 -- 172.21.15.79:0/4148958241 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20800b9c0 con 0x7ff21006a460 2023-06-08T03:50:57.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff20e7fc700 1 -- 172.21.15.79:0/4148958241 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800bd70 con 0x7ff21006a460 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 -- 172.21.15.79:0/4148958241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 msgr2=0x7ff2100cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 --2- 172.21.15.79:0/4148958241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100cd190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff208009e50 tx=0x7ff20800b330 comp rx=0 tx=0).stop 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 -- 172.21.15.79:0/4148958241 shutdown_connections 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 --2- 172.21.15.79:0/4148958241 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100cd190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 -- 172.21.15.79:0/4148958241 >> 172.21.15.79:0/4148958241 conn(0x7ff21006d140 msgr2=0x7ff21006d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 -- 172.21.15.79:0/4148958241 shutdown_connections 2023-06-08T03:50:57.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.080+0000 7ff2161a8700 1 -- 172.21.15.79:0/4148958241 wait complete. 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.081+0000 7ff2161a8700 1 Processor -- start 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 -- start start 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100715e0 0x7ff2101f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2100cb2e0 0x7ff2100c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2101e84c0 con 0x7ff21006a460 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2101560f0 con 0x7ff2100715e0 2023-06-08T03:50:57.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff2161a8700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100ae6d0 con 0x7ff2100cb2e0 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff20f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff20f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35684/0 (socket says 172.21.15.79:35684) 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.082+0000 7ff20f7fe700 1 -- 172.21.15.79:0/280168208 learned_addr learned my addr 172.21.15.79:0/280168208 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20effd700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100715e0 0x7ff2101f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20ffff700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2100cb2e0 0x7ff2100c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20f7fe700 1 -- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2100cb2e0 msgr2=0x7ff2100c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20f7fe700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2100cb2e0 0x7ff2100c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20f7fe700 1 -- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100715e0 msgr2=0x7ff2101f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20f7fe700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100715e0 0x7ff2101f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20f7fe700 1 -- 172.21.15.79:0/280168208 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff208009b30 con 0x7ff21006a460 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20effd700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100715e0 0x7ff2101f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff20f7fe700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100ae190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff20400c4e0 tx=0x7ff20400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2040052d0 con 0x7ff21006a460 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204005430 con 0x7ff21006a460 2023-06-08T03:50:57.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2040038c0 con 0x7ff21006a460 2023-06-08T03:50:57.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2101e81e0 con 0x7ff21006a460 2023-06-08T03:50:57.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.083+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2100ac6e0 con 0x7ff21006a460 2023-06-08T03:50:57.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.084+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff210073e90 con 0x7ff21006a460 2023-06-08T03:50:57.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.084+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff204003a20 con 0x7ff21006a460 2023-06-08T03:50:57.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.085+0000 7ff1f7fff700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1fc06ace0 0x7ff1fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:57.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.085+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff204088c00 con 0x7ff21006a460 2023-06-08T03:50:57.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.085+0000 7ff20effd700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1fc06ace0 0x7ff1fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:57.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.086+0000 7ff20effd700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1fc06ace0 0x7ff1fc06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff2101e88a0 tx=0x7ff200007ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:57.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.089+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff20408f080 con 0x7ff21006a460 2023-06-08T03:50:57.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.319+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2101e8910 con 0x7ff21006a460 2023-06-08T03:50:57.322 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.319+0000 7ff1f7fff700 1 -- 172.21.15.79:0/280168208 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff204017070 con 0x7ff21006a460 2023-06-08T03:50:57.323 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-08T03:50:57.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1fc06ace0 msgr2=0x7ff1fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1fc06ace0 0x7ff1fc06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff2101e88a0 tx=0x7ff200007ce0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 msgr2=0x7ff2100ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100ae190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff20400c4e0 tx=0x7ff20400c8a0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 shutdown_connections 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1fc06ace0 0x7ff1fc06d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2100cb2e0 0x7ff2100c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff21006a460 0x7ff2100ae190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 --2- 172.21.15.79:0/280168208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2100715e0 0x7ff2101f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:57.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 >> 172.21.15.79:0/280168208 conn(0x7ff21006d140 msgr2=0x7ff21006f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:57.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.321+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 shutdown_connections 2023-06-08T03:50:57.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:57.322+0000 7ff2161a8700 1 -- 172.21.15.79:0/280168208 wait complete. 2023-06-08T03:50:57.336 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-08T03:50:57.336 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-08T03:50:57.337 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-08T03:50:57.337 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-06-08T03:50:57.337 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-08T03:50:57.337 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-08T03:50:57.337 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-08T03:50:57.338 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-08T03:50:57.338 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-06-08T03:50:57.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.338 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-08T03:50:57.355 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:50:57.355 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:57.356 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42330sec preferred_lft 42330sec 2023-06-08T03:50:57.357 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::6d1d:ad0f:719a:51d1/64 scope link noprefixroute 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 16:7f:25:be:e6:a2 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::7883:54ff:fea8:cae8/64 scope link 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.358 INFO:teuthology.orchestra.run.smithi079.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-08T03:50:57.359 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 16:7f:25:be:e6:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-08T03:50:57.359 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::147f:25ff:febe:e6a2/64 scope link 2023-06-08T03:50:57.359 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.359 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-08T03:50:57.420 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-08T03:50:57.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.422 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-08T03:50:57.480 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-08T03:50:57.480 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-08T03:50:57.480 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-08T03:50:57.480 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-08T03:50:57.480 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-08T03:50:57.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.481 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:57.481 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:57.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-08T03:50:57.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-08T03:50:57.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-08T03:50:57.482 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:57.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.585 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-08T03:50:57.601 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-08T03:50:57.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.602 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2023-06-08T03:50:57.657 INFO:teuthology.orchestra.run.smithi079.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-06-08T03:50:57.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.658 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:57.658 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:57.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-08T03:50:57.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-06-08T03:50:57.659 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:57.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.750 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-08T03:50:57.787 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:50:57.788 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether b2:23:00:d0:96:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-08T03:50:57.789 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-08T03:50:57.789 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.789 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::b023:ff:fed0:962d/64 scope link 2023-06-08T03:50:57.789 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-08T03:50:57.876 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:50:57.877 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether b2:23:00:d0:96:2d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-08T03:50:57.878 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-08T03:50:57.878 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.878 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::b023:ff:fed0:962d/64 scope link 2023-06-08T03:50:57.878 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:57.889 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-06-08T03:50:57.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:57.889 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:57.890 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:57.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-06-08T03:50:57.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-08T03:50:57.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-06-08T03:50:57.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-06-08T03:50:57.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-06-08T03:50:57.891 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:58.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:58.137 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:50:58.138 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:50:58.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.1 up 2023-06-08T03:50:58.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.1 master ceph-brx 2023-06-08T03:50:58.138 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:50:58.249 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-06-08T03:50:58.249 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:50:58.249 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-06-08T03:50:58.269 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-06-08T03:50:58.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:50:58.269 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-06-08T03:50:58.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:50:58.391 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:50:58.391 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:50:58.456 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:50:58.458 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:50:58.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:58.459 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:50:58.514 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-08T03:50:58.515 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-06-08T03:50:58.515 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-06-08T03:50:58.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:50:58.582 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:50:58.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:50:58.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi079.stderr:2023-06-08T03:50:58.639+0000 7f8798ec85c0 -1 init, newargv = 0x564e5271b9f0 newargc=15 2023-06-08T03:50:58.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi079.stderr:ceph-fuse[112040]: starting ceph client 2023-06-08T03:50:58.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi079.stderr:ceph-fuse[112040]: starting fuse 2023-06-08T03:50:58.651 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:50:58.652 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:50:58.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:58.653 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:50:58.709 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-08T03:50:58.709 INFO:teuthology.orchestra.run.smithi079.stdout:48 2023-06-08T03:50:58.709 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-06-08T03:50:58.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-08T03:50:58.710 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-08T03:50:58.710 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-08T03:50:58.710 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-08T03:50:58.710 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-08T03:50:58.711 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:58.711 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-08T03:50:58.711 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-06-08T03:50:58.711 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-08T03:50:58.712 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-06-08T03:50:58.712 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:58.712 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-08T03:50:58.712 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-08T03:50:58.712 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-08T03:50:58.713 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-08T03:50:58.713 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:58.713 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-08T03:50:58.713 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-08T03:50:58.714 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-08T03:50:58.714 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-08T03:50:58.714 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-08T03:50:58.714 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-08T03:50:58.714 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-08T03:50:58.715 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-08T03:50:58.715 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:50:58.715 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.2")) 2023-06-08T03:50:58.715 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-08T03:50:58.830 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.2.112040.asok 2023-06-08T03:50:59.012 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.112040.asok 2023-06-08T03:50:59.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:59.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.112040.asok status 2023-06-08T03:50:59.168 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:50:59.168 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-08T03:50:59.168 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "a02638f808b8fe11bccdd01222c0e6e2eb71120b", 2023-06-08T03:50:59.168 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)", 2023-06-08T03:50:59.168 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "2", 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "112040", 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-08T03:50:59.169 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4887, 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 4887 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:50:59.170 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.2:0", 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2712151383 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.2:0", 2023-06-08T03:50:59.171 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 2712151383 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.4887 192.168.0.2:0/2712151383", 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.2:0/2712151383", 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 9, 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 25, 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-08T03:50:59.172 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-08T03:50:59.173 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-08T03:50:59.173 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:50:59.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-08T03:50:59.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.294+0000 7f907d72b700 1 Processor -- start 2023-06-08T03:50:59.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.294+0000 7f907d72b700 1 -- start start 2023-06-08T03:50:59.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f907d72b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f907806bb30 0x7f907806f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:59.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f907d72b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f90780b0fe0 0x7f90780b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:59.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f907d72b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907806edd0 con 0x7f907815c680 2023-06-08T03:50:59.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f907d72b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907806f7a0 con 0x7f907806bb30 2023-06-08T03:50:59.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f907d72b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90780b1960 con 0x7f90780b0fe0 2023-06-08T03:50:59.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f90767fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f907806bb30 0x7f907806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:59.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f9076ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f907815c680 0x7f90780ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59182/0 (socket says 172.21.15.79:59182) 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f9076ffd700 1 -- 172.21.15.79:0/2185559153 learned_addr learned my addr 172.21.15.79:0/2185559153 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.295+0000 7f90777fe700 1 --2- 172.21.15.79:0/2185559153 >> v2:172.21.15.156:3300/0 conn(0x7f90780b0fe0 0x7f90780b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f90767fc700 1 -- 172.21.15.79:0/2185559153 >> v2:172.21.15.156:3300/0 conn(0x7f90780b0fe0 msgr2=0x7f90780b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f90767fc700 1 --2- 172.21.15.79:0/2185559153 >> v2:172.21.15.156:3300/0 conn(0x7f90780b0fe0 0x7f90780b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f90767fc700 1 -- 172.21.15.79:0/2185559153 >> v1:172.21.15.79:6789/0 conn(0x7f907815c680 legacy=0x7f90780ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f9075ffb700 1 -- 172.21.15.79:0/2185559153 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331150933 0 0) 0x7f907806edd0 con 0x7f907815c680 2023-06-08T03:50:59.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f90767fc700 1 -- 172.21.15.79:0/2185559153 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907806eaa0 con 0x7f907806bb30 2023-06-08T03:50:59.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f90767fc700 1 --2- 172.21.15.79:0/2185559153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f907806bb30 0x7f907806f260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90780b5490 tx=0x7f9068009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff66a8c65c0e34fe server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:59.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f9075ffb700 1 -- 172.21.15.79:0/2185559153 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068011830 con 0x7f907806bb30 2023-06-08T03:50:59.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f9075ffb700 1 -- 172.21.15.79:0/2185559153 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068011990 con 0x7f907806bb30 2023-06-08T03:50:59.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f9075ffb700 1 -- 172.21.15.79:0/2185559153 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068011d20 con 0x7f907806bb30 2023-06-08T03:50:59.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 -- 172.21.15.79:0/2185559153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f907806bb30 msgr2=0x7f907806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:59.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 --2- 172.21.15.79:0/2185559153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f907806bb30 0x7f907806f260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f90780b5490 tx=0x7f9068009b20 comp rx=0 tx=0).stop 2023-06-08T03:50:59.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 -- 172.21.15.79:0/2185559153 shutdown_connections 2023-06-08T03:50:59.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 --2- 172.21.15.79:0/2185559153 >> v2:172.21.15.156:3300/0 conn(0x7f90780b0fe0 0x7f90780b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 --2- 172.21.15.79:0/2185559153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f907806bb30 0x7f907806f260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 -- 172.21.15.79:0/2185559153 >> 172.21.15.79:0/2185559153 conn(0x7f90780cbfe0 msgr2=0x7f90780cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.296+0000 7f907d72b700 1 -- 172.21.15.79:0/2185559153 shutdown_connections 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.297+0000 7f907d72b700 1 -- 172.21.15.79:0/2185559153 wait complete. 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.298+0000 7f907d72b700 1 Processor -- start 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.298+0000 7f907d72b700 1 -- start start 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f907d72b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907806bb30 0x7f90780ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:59.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f907d72b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 0x7f90781ef060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f907d72b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f907815c680 0x7f90781ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f907d72b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907815fa20 con 0x7f90780b0fe0 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f907d72b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781efae0 con 0x7f907815c680 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f907d72b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907815e6c0 con 0x7f907806bb30 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f90767fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 0x7f90781ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f90767fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 0x7f90781ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35714/0 (socket says 172.21.15.79:35714) 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f90767fc700 1 -- 172.21.15.79:0/3580307036 learned_addr learned my addr 172.21.15.79:0/3580307036 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:50:59.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f9076ffd700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907806bb30 0x7f90780ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.299+0000 7f90777fe700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f907815c680 0x7f90781ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90767fc700 1 -- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907806bb30 msgr2=0x7f90780ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90767fc700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907806bb30 0x7f90780ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90767fc700 1 -- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f907815c680 msgr2=0x7f90781ef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90767fc700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f907815c680 0x7f90781ef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90767fc700 1 -- 172.21.15.79:0/3580307036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90680094a0 con 0x7f90780b0fe0 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90777fe700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f907815c680 0x7f90781ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:50:59.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f90767fc700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 0x7f90781ef060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f90680090b0 tx=0x7f9068004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:59.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680036a0 con 0x7f90780b0fe0 2023-06-08T03:50:59.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906801f070 con 0x7f90780b0fe0 2023-06-08T03:50:59.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90780ab960 con 0x7f90780b0fe0 2023-06-08T03:50:59.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068003a00 con 0x7f90780b0fe0 2023-06-08T03:50:59.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.300+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907815ec40 con 0x7f90780b0fe0 2023-06-08T03:50:59.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.301+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9078073e90 con 0x7f90780b0fe0 2023-06-08T03:50:59.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.301+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9068003b60 con 0x7f90780b0fe0 2023-06-08T03:50:59.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.302+0000 7f905f7fe700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f906406ace0 0x7f906406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:50:59.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.302+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9068021070 con 0x7f90780b0fe0 2023-06-08T03:50:59.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.302+0000 7f9076ffd700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f906406ace0 0x7f906406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:50:59.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.302+0000 7f9076ffd700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f906406ace0 0x7f906406d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f906c000f60 tx=0x7f906c007e70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:50:59.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.306+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9068055e40 con 0x7f90780b0fe0 2023-06-08T03:50:59.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.538+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f90780b54e0 con 0x7f90780b0fe0 2023-06-08T03:50:59.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.539+0000 7f905f7fe700 1 -- 172.21.15.79:0/3580307036 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9068015070 con 0x7f90780b0fe0 2023-06-08T03:50:59.542 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-08T03:50:59.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.540+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f906406ace0 msgr2=0x7f906406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:59.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.540+0000 7f907d72b700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f906406ace0 0x7f906406d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f906c000f60 tx=0x7f906c007e70 comp rx=0 tx=0).stop 2023-06-08T03:50:59.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.540+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 msgr2=0x7f90781ef060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:50:59.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.540+0000 7f907d72b700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 0x7f90781ef060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f90680090b0 tx=0x7f9068004750 comp rx=0 tx=0).stop 2023-06-08T03:50:59.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 shutdown_connections 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f907815c680 0x7f90781ef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f906406ace0 0x7f906406d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f907806bb30 0x7f90780ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 --2- 172.21.15.79:0/3580307036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90780b0fe0 0x7f90781ef060 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 >> 172.21.15.79:0/3580307036 conn(0x7f90780cbfe0 msgr2=0x7f907815ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 shutdown_connections 2023-06-08T03:50:59.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:50:59.541+0000 7f907d72b700 1 -- 172.21.15.79:0/3580307036 wait complete. 2023-06-08T03:50:59.555 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-08T03:50:59.555 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-08T03:50:59.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-06-08T03:50:59.556 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-08T03:50:59.556 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-08T03:50:59.556 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-08T03:50:59.556 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-08T03:50:59.557 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-08T03:50:59.557 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-08T03:50:59.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:59.557 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-06-08T03:50:59.578 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:59.579 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-06-08T03:50:59.580 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-08T03:50:59.581 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42396sec preferred_lft 42396sec 2023-06-08T03:50:59.581 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::6655:17d8:e24f:6e2/64 scope link noprefixroute 2023-06-08T03:50:59.581 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:50:59.581 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-08T03:50:59.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:59.582 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:50:59.582 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:50:59.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add name ceph-brx type bridge 2023-06-08T03:50:59.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr flush dev ceph-brx 2023-06-08T03:50:59.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set ceph-brx up 2023-06-08T03:50:59.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-08T03:50:59.583 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:50:59.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:59.773 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-08T03:50:59.801 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-06-08T03:50:59.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:59.802 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-06-08T03:50:59.879 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-06-08T03:50:59.879 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-08T03:50:59.879 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-08T03:50:59.880 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-08T03:50:59.880 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-08T03:50:59.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:50:59.880 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:50:59.881 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:50:59.881 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-08T03:50:59.881 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-08T03:50:59.881 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-08T03:50:59.882 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:00.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:00.147 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-06-08T03:51:00.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:00.163 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-06-08T03:51:00.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:00.220 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:00.221 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:00.221 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-08T03:51:00.221 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-06-08T03:51:00.221 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:00.315 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-06-08T03:51:00.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:00.315 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:00.316 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:00.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-06-08T03:51:00.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-08T03:51:00.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-06-08T03:51:00.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-06-08T03:51:00.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-06-08T03:51:00.317 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:00.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:00.535 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:00.535 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:00.535 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-06-08T03:51:00.535 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.0 master ceph-brx 2023-06-08T03:51:00.536 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:00.637 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-06-08T03:51:00.637 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:51:00.638 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-08T03:51:00.658 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-08T03:51:00.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:51:00.658 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-08T03:51:00.714 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2023-06-08T03:51:00.782 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:51:00.782 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:51:00.848 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:51:00.849 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:51:00.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:00.850 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2023-06-08T03:51:00.906 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-08T03:51:00.907 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-06-08T03:51:00.949 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2023-06-08T03:51:00.975 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:51:00.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:51:01.040 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:51:01.041 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:51:01.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:01.042 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2023-06-08T03:51:01.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi156.stderr:2023-06-08T03:51:01.057+0000 7f5240cdf5c0 -1 init, newargv = 0x5623d29e98b0 newargc=15 2023-06-08T03:51:01.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi156.stderr:ceph-fuse[105629]: starting ceph client 2023-06-08T03:51:01.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi156.stderr:ceph-fuse[105629]: starting fuse 2023-06-08T03:51:01.077 INFO:teuthology.orchestra.run.smithi156.stdout:47 2023-06-08T03:51:01.077 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-06-08T03:51:01.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-08T03:51:01.078 DEBUG:teuthology.orchestra.run.smithi156:> import glob 2023-06-08T03:51:01.078 DEBUG:teuthology.orchestra.run.smithi156:> import re 2023-06-08T03:51:01.078 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-06-08T03:51:01.079 DEBUG:teuthology.orchestra.run.smithi156:> import subprocess 2023-06-08T03:51:01.079 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:01.079 DEBUG:teuthology.orchestra.run.smithi156:> def _find_admin_socket(client_name): 2023-06-08T03:51:01.079 DEBUG:teuthology.orchestra.run.smithi156:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-06-08T03:51:01.079 DEBUG:teuthology.orchestra.run.smithi156:> files = glob.glob(asok_path) 2023-06-08T03:51:01.080 DEBUG:teuthology.orchestra.run.smithi156:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-06-08T03:51:01.080 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:01.080 DEBUG:teuthology.orchestra.run.smithi156:> # Given a non-glob path, it better be there 2023-06-08T03:51:01.080 DEBUG:teuthology.orchestra.run.smithi156:> if "*" not in asok_path: 2023-06-08T03:51:01.081 DEBUG:teuthology.orchestra.run.smithi156:> assert(len(files) == 1) 2023-06-08T03:51:01.081 DEBUG:teuthology.orchestra.run.smithi156:> return files[0] 2023-06-08T03:51:01.081 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:01.081 DEBUG:teuthology.orchestra.run.smithi156:> for f in files: 2023-06-08T03:51:01.081 DEBUG:teuthology.orchestra.run.smithi156:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-08T03:51:01.082 DEBUG:teuthology.orchestra.run.smithi156:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-08T03:51:01.082 DEBUG:teuthology.orchestra.run.smithi156:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-08T03:51:01.082 DEBUG:teuthology.orchestra.run.smithi156:> contents = proc_f.read() 2023-06-08T03:51:01.082 DEBUG:teuthology.orchestra.run.smithi156:> if mountpoint in contents: 2023-06-08T03:51:01.083 DEBUG:teuthology.orchestra.run.smithi156:> return f 2023-06-08T03:51:01.083 DEBUG:teuthology.orchestra.run.smithi156:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-08T03:51:01.083 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:01.083 DEBUG:teuthology.orchestra.run.smithi156:> print(_find_admin_socket("client.1")) 2023-06-08T03:51:01.083 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-06-08T03:51:01.201 INFO:teuthology.orchestra.run.smithi156.stdout:/var/run/ceph/ceph-client.1.105629.asok 2023-06-08T03:51:01.383 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.105629.asok 2023-06-08T03:51:01.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:01.384 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.105629.asok status 2023-06-08T03:51:01.539 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-06-08T03:51:01.539 INFO:teuthology.orchestra.run.smithi156.stdout: "metadata": { 2023-06-08T03:51:01.539 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_sha1": "a02638f808b8fe11bccdd01222c0e6e2eb71120b", 2023-06-08T03:51:01.539 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)", 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: "entity_id": "1", 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: "hostname": "smithi156", 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: "pid": "105629", 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: "root": "/" 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:01.540 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_count": 0, 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_pinned_count": 0, 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4631, 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "inst": { 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "name": { 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "client", 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "num": 4631 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:01.541 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 2295012494 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2023-06-08T03:51:01.542 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 2295012494 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "inst_str": "client.4631 192.168.0.1:0/2295012494", 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "addr_str": "192.168.0.1:0/2295012494", 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "inode_count": 1, 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_epoch": 9, 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch": 25, 2023-06-08T03:51:01.543 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch_barrier": 0, 2023-06-08T03:51:01.544 INFO:teuthology.orchestra.run.smithi156.stdout: "blocklisted": false, 2023-06-08T03:51:01.544 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs" 2023-06-08T03:51:01.544 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-06-08T03:51:01.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-08T03:51:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.678+0000 7ff1e7f3c700 1 Processor -- start 2023-06-08T03:51:01.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.678+0000 7ff1e7f3c700 1 -- start start 2023-06-08T03:51:01.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e7f3c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e006a460 0x7ff1e0071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:01.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e7f3c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff1e00cd6c0 0x7ff1e0071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:01.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e7f3c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00cee20 con 0x7ff1e015f6c0 2023-06-08T03:51:01.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e7f3c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e00cf330 con 0x7ff1e00cd6c0 2023-06-08T03:51:01.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e7f3c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e006fa00 con 0x7ff1e006a460 2023-06-08T03:51:01.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e64d9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7ff1e00cd6c0 0x7ff1e0071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e5cd8700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff1e015f6c0 0x7ff1e0069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59196/0 (socket says 172.21.15.79:59196) 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.679+0000 7ff1e5cd8700 1 -- 172.21.15.79:0/3735249129 learned_addr learned my addr 172.21.15.79:0/3735249129 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e54d7700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e006a460 0x7ff1e0071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e64d9700 1 -- 172.21.15.79:0/3735249129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e006a460 msgr2=0x7ff1e0071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e64d9700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e006a460 0x7ff1e0071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e64d9700 1 -- 172.21.15.79:0/3735249129 >> v1:172.21.15.79:6789/0 conn(0x7ff1e015f6c0 legacy=0x7ff1e0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:01.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e64d9700 1 -- 172.21.15.79:0/3735249129 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00cd1e0 con 0x7ff1e00cd6c0 2023-06-08T03:51:01.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e54d7700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e006a460 0x7ff1e0071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:01.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e64d9700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e0071590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009d90 tx=0x7ff1d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f6e8a5546e52a2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:01.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e4cd6700 1 -- 172.21.15.79:0/3735249129 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800e070 con 0x7ff1e00cd6c0 2023-06-08T03:51:01.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e4cd6700 1 -- 172.21.15.79:0/3735249129 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d800bbd0 con 0x7ff1e00cd6c0 2023-06-08T03:51:01.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.680+0000 7ff1e4cd6700 1 -- 172.21.15.79:0/3735249129 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8012430 con 0x7ff1e00cd6c0 2023-06-08T03:51:01.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/3735249129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e00cd6c0 msgr2=0x7ff1e0071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e0071590 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009d90 tx=0x7ff1d800b330 comp rx=0 tx=0).stop 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/3735249129 shutdown_connections 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e0071590 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/3735249129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e006a460 0x7ff1e0071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/3735249129 >> 172.21.15.79:0/3735249129 conn(0x7ff1e00b1d20 msgr2=0x7ff1e006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/3735249129 shutdown_connections 2023-06-08T03:51:01.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.681+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/3735249129 wait complete. 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.682+0000 7ff1e7f3c700 1 Processor -- start 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.682+0000 7ff1e7f3c700 1 -- start start 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 0x7ff1e015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e00c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e015f6c0 0x7ff1e01f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00ae9b0 con 0x7ff1e006a460 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00ae110 con 0x7ff1e00cd6c0 2023-06-08T03:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00ae3f0 con 0x7ff1e015f6c0 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 0x7ff1e015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 0x7ff1e015a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35726/0 (socket says 172.21.15.79:35726) 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 -- 172.21.15.79:0/564106154 learned_addr learned my addr 172.21.15.79:0/564106154 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e64d9700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e015f6c0 0x7ff1e01f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e54d7700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e00c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 -- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e015f6c0 msgr2=0x7ff1e01f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:01.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e015f6c0 0x7ff1e01f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 -- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e00cd6c0 msgr2=0x7ff1e00c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e00c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 -- 172.21.15.79:0/564106154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8009a70 con 0x7ff1e006a460 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e64d9700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e015f6c0 0x7ff1e01f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e5cd8700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 0x7ff1e015a870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c480 tx=0x7ff1d000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0005d70 con 0x7ff1e006a460 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00070d0 con 0x7ff1e006a460 2023-06-08T03:51:01.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e00ae6d0 con 0x7ff1e006a460 2023-06-08T03:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0007440 con 0x7ff1e006a460 2023-06-08T03:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.683+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e01e70d0 con 0x7ff1e006a460 2023-06-08T03:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.684+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e01e6010 con 0x7ff1e006a460 2023-06-08T03:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.685+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1d00075a0 con 0x7ff1e006a460 2023-06-08T03:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.685+0000 7ff1d67fc700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1c406ace0 0x7ff1c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.685+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff1d0088bd0 con 0x7ff1e006a460 2023-06-08T03:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.685+0000 7ff1e54d7700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1c406ace0 0x7ff1c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.686+0000 7ff1e54d7700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1c406ace0 0x7ff1c406d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff1e015b220 tx=0x7ff1dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.690+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff1e01e6010 con 0x7ff1e006a460 2023-06-08T03:51:01.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:51:01.882 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:51:01.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.923+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff1e01e6010 con 0x7ff1e006a460 2023-06-08T03:51:01.926 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.923+0000 7ff1d67fc700 1 -- 172.21.15.79:0/564106154 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff1e01e6010 con 0x7ff1e006a460 2023-06-08T03:51:01.926 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-06-08T03:51:01.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1c406ace0 msgr2=0x7ff1c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:01.928 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1c406ace0 0x7ff1c406d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff1e015b220 tx=0x7ff1dc008040 comp rx=0 tx=0).stop 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 msgr2=0x7ff1e015a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 0x7ff1e015a870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c480 tx=0x7ff1d000c840 comp rx=0 tx=0).stop 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 shutdown_connections 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff1c406ace0 0x7ff1c406d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff1e015f6c0 0x7ff1e01f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1e006a460 0x7ff1e015a870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.929 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 --2- 172.21.15.79:0/564106154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff1e00cd6c0 0x7ff1e00c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:01.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 >> 172.21.15.79:0/564106154 conn(0x7ff1e00b1d20 msgr2=0x7ff1e00cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:01.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 shutdown_connections 2023-06-08T03:51:01.930 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:01.925+0000 7ff1e7f3c700 1 -- 172.21.15.79:0/564106154 wait complete. 2023-06-08T03:51:01.940 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-08T03:51:01.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-08T03:51:01.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-06-08T03:51:01.940 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-06-08T03:51:01.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:01.941 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-08T03:51:01.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-08T03:51:01.941 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-08T03:51:01.941 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-06-08T03:51:01.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:01.942 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-06-08T03:51:01.999 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:51:02.000 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:02.001 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42393sec preferred_lft 42393sec 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::6655:17d8:e24f:6e2/64 scope link noprefixroute 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether ba:50:a3:44:8d:34 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.002 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a428:1cff:fed8:a271/64 scope link 2023-06-08T03:51:02.003 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.003 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-08T03:51:02.003 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether ba:50:a3:44:8d:34 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-08T03:51:02.003 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::b850:a3ff:fe44:8d34/64 scope link 2023-06-08T03:51:02.003 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.004 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-08T03:51:02.064 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-06-08T03:51:02.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.066 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-06-08T03:51:02.124 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-06-08T03:51:02.125 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-08T03:51:02.125 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-08T03:51:02.125 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-08T03:51:02.125 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-08T03:51:02.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.126 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:02.126 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:02.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-08T03:51:02.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-08T03:51:02.127 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-08T03:51:02.127 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:02.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.229 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-06-08T03:51:02.245 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-08T03:51:02.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.246 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-06-08T03:51:02.301 INFO:teuthology.orchestra.run.smithi156.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-06-08T03:51:02.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.302 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:02.302 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:02.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-08T03:51:02.303 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-06-08T03:51:02.303 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:02.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.395 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-08T03:51:02.434 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:51:02.434 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:51:02.434 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:51:02.434 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.434 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-06-08T03:51:02.434 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.435 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:51:02.435 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 4a:2b:7c:ca:33:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-08T03:51:02.435 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-08T03:51:02.435 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.435 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::482b:7cff:feca:331b/64 scope link 2023-06-08T03:51:02.435 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.445 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-08T03:51:02.527 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:51:02.528 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:51:02.528 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:51:02.528 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.528 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-06-08T03:51:02.528 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.528 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:51:02.529 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 4a:2b:7c:ca:33:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-08T03:51:02.529 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-08T03:51:02.529 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.529 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::482b:7cff:feca:331b/64 scope link 2023-06-08T03:51:02.529 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:02.540 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-06-08T03:51:02.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.540 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:02.541 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:02.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-06-08T03:51:02.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-08T03:51:02.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-06-08T03:51:02.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-06-08T03:51:02.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-06-08T03:51:02.542 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:02.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:02.805 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-06-08T03:51:02.805 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-06-08T03:51:02.805 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.1 up 2023-06-08T03:51:02.806 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.1 master ceph-brx 2023-06-08T03:51:02.806 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-06-08T03:51:02.913 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-06-08T03:51:02.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:51:02.914 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:02.932 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-06-08T03:51:02.933 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:51:02.933 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:02.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2023-06-08T03:51:03.055 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:51:03.055 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:51:03.120 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:51:03.121 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:51:03.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.122 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2023-06-08T03:51:03.178 INFO:teuthology.orchestra.run.smithi156.stdout:47 2023-06-08T03:51:03.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-06-08T03:51:03.179 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-06-08T03:51:03.221 DEBUG:teuthology.orchestra.run.smithi156:> sudo modprobe fuse 2023-06-08T03:51:03.245 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:51:03.246 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:51:03.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi156.stderr:2023-06-08T03:51:03.304+0000 7f7a60cc35c0 -1 init, newargv = 0x56438f439660 newargc=15 2023-06-08T03:51:03.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi156.stderr:ceph-fuse[106237]: starting ceph client 2023-06-08T03:51:03.312 INFO:teuthology.orchestra.run.smithi156.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:51:03.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi156.stderr:ceph-fuse[106237]: starting fuse 2023-06-08T03:51:03.314 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:51:03.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.314 DEBUG:teuthology.orchestra.run.smithi156:> ls /sys/fs/fuse/connections 2023-06-08T03:51:03.370 INFO:teuthology.orchestra.run.smithi156.stdout:47 2023-06-08T03:51:03.370 INFO:teuthology.orchestra.run.smithi156.stdout:48 2023-06-08T03:51:03.370 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-06-08T03:51:03.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-08T03:51:03.371 DEBUG:teuthology.orchestra.run.smithi156:> import glob 2023-06-08T03:51:03.371 DEBUG:teuthology.orchestra.run.smithi156:> import re 2023-06-08T03:51:03.371 DEBUG:teuthology.orchestra.run.smithi156:> import os 2023-06-08T03:51:03.372 DEBUG:teuthology.orchestra.run.smithi156:> import subprocess 2023-06-08T03:51:03.372 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:03.372 DEBUG:teuthology.orchestra.run.smithi156:> def _find_admin_socket(client_name): 2023-06-08T03:51:03.372 DEBUG:teuthology.orchestra.run.smithi156:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-06-08T03:51:03.373 DEBUG:teuthology.orchestra.run.smithi156:> files = glob.glob(asok_path) 2023-06-08T03:51:03.373 DEBUG:teuthology.orchestra.run.smithi156:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-06-08T03:51:03.373 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:03.373 DEBUG:teuthology.orchestra.run.smithi156:> # Given a non-glob path, it better be there 2023-06-08T03:51:03.374 DEBUG:teuthology.orchestra.run.smithi156:> if "*" not in asok_path: 2023-06-08T03:51:03.374 DEBUG:teuthology.orchestra.run.smithi156:> assert(len(files) == 1) 2023-06-08T03:51:03.374 DEBUG:teuthology.orchestra.run.smithi156:> return files[0] 2023-06-08T03:51:03.374 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:03.374 DEBUG:teuthology.orchestra.run.smithi156:> for f in files: 2023-06-08T03:51:03.375 DEBUG:teuthology.orchestra.run.smithi156:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-08T03:51:03.375 DEBUG:teuthology.orchestra.run.smithi156:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-08T03:51:03.375 DEBUG:teuthology.orchestra.run.smithi156:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-08T03:51:03.375 DEBUG:teuthology.orchestra.run.smithi156:> contents = proc_f.read() 2023-06-08T03:51:03.376 DEBUG:teuthology.orchestra.run.smithi156:> if mountpoint in contents: 2023-06-08T03:51:03.376 DEBUG:teuthology.orchestra.run.smithi156:> return f 2023-06-08T03:51:03.376 DEBUG:teuthology.orchestra.run.smithi156:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-08T03:51:03.376 DEBUG:teuthology.orchestra.run.smithi156:> 2023-06-08T03:51:03.377 DEBUG:teuthology.orchestra.run.smithi156:> print(_find_admin_socket("client.3")) 2023-06-08T03:51:03.377 DEBUG:teuthology.orchestra.run.smithi156:> ' 2023-06-08T03:51:03.491 INFO:teuthology.orchestra.run.smithi156.stdout:/var/run/ceph/ceph-client.3.106237.asok 2023-06-08T03:51:03.674 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.106237.asok 2023-06-08T03:51:03.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.106237.asok status 2023-06-08T03:51:03.830 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "metadata": { 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_sha1": "a02638f808b8fe11bccdd01222c0e6e2eb71120b", 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)", 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "entity_id": "3", 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "hostname": "smithi156", 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-06-08T03:51:03.831 INFO:teuthology.orchestra.run.smithi156.stdout: "pid": "106237", 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "root": "/" 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_count": 0, 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "dentry_pinned_count": 0, 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "id": 4765, 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "inst": { 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "name": { 2023-06-08T03:51:03.832 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "client", 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: "num": 4765 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.2:0", 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 4221403452 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-06-08T03:51:03.833 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": { 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "type": "v1", 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "addr": "192.168.0.2:0", 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "nonce": 4221403452 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "inst_str": "client.4765 192.168.0.2:0/4221403452", 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "addr_str": "192.168.0.2:0/4221403452", 2023-06-08T03:51:03.834 INFO:teuthology.orchestra.run.smithi156.stdout: "inode_count": 1, 2023-06-08T03:51:03.835 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_epoch": 9, 2023-06-08T03:51:03.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch": 25, 2023-06-08T03:51:03.835 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_epoch_barrier": 0, 2023-06-08T03:51:03.835 INFO:teuthology.orchestra.run.smithi156.stdout: "blocklisted": false, 2023-06-08T03:51:03.835 INFO:teuthology.orchestra.run.smithi156.stdout: "fs_name": "cephfs" 2023-06-08T03:51:03.835 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-06-08T03:51:03.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.840 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-08T03:51:03.840 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:03.861 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-08T03:51:03.861 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:03.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:03.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.933 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-08T03:51:03.934 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.2 2023-06-08T03:51:03.993 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-08T03:51:03.993 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-06-08T03:51:03.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:03.993 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-06-08T03:51:04.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:04.062 DEBUG:teuthology.orchestra.run.smithi156:> stat --file-system '--printf=%T 2023-06-08T03:51:04.062 DEBUG:teuthology.orchestra.run.smithi156:> ' -- /home/ubuntu/cephtest/mnt.1 2023-06-08T03:51:04.079 INFO:teuthology.orchestra.run.smithi156.stdout:fuseblk 2023-06-08T03:51:04.079 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-06-08T03:51:04.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:04.080 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-08T03:51:04.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:04.146 DEBUG:teuthology.orchestra.run.smithi156:> stat --file-system '--printf=%T 2023-06-08T03:51:04.146 DEBUG:teuthology.orchestra.run.smithi156:> ' -- /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:04.203 INFO:teuthology.orchestra.run.smithi156.stdout:fuseblk 2023-06-08T03:51:04.203 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:04.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:04.203 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:04.270 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-06-08T03:51:04.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:04.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.411+0000 7f5a6ed1a700 1 Processor -- start 2023-06-08T03:51:04.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.411+0000 7f5a6ed1a700 1 -- start start 2023-06-08T03:51:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.411+0000 7f5a6ed1a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a68158b20 0x7f5a68070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.411+0000 7f5a6ed1a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 0x7f5a680c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.411+0000 7f5a6ed1a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a680c8b40 con 0x7f5a68158ef0 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.411+0000 7f5a6ed1a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a680cb980 con 0x7f5a68158b20 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a6ed1a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680c8de0 con 0x7f5a680b06c0 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a67fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5a68158ef0 0x7f5a680affb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59210/0 (socket says 172.21.15.79:59210) 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a67fff700 1 -- 172.21.15.79:0/1485078343 learned_addr learned my addr 172.21.15.79:0/1485078343 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535186019 0 0) 0x7f5a680c8b40 con 0x7f5a68158ef0 2023-06-08T03:51:04.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/1485078343 >> v2:172.21.15.156:3300/0 conn(0x7f5a68158b20 0x7f5a68070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a6d2b7700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 0x7f5a680c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a580035b0 con 0x7f5a68158ef0 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 491299305 0 0) 0x7f5a580035b0 con 0x7f5a68158ef0 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 msgr2=0x7f5a680c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 0x7f5a680c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 >> v2:172.21.15.156:3300/0 conn(0x7f5a68158b20 msgr2=0x7f5a68070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 --2- 172.21.15.79:0/1485078343 >> v2:172.21.15.156:3300/0 conn(0x7f5a68158b20 0x7f5a68070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680c90c0 con 0x7f5a68158ef0 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f5a54001670 con 0x7f5a68158ef0 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 >> v1:172.21.15.79:6789/0 conn(0x7f5a68158ef0 legacy=0x7f5a680affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a58003a70 0x7f5a58003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a58004850 0x7f5a58004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 0x7f5a58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680cb980 con 0x7f5a58003a70 2023-06-08T03:51:04.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a680c8de0 con 0x7f5a58004850 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5800ba10 con 0x7f5a680b06c0 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a58003a70 0x7f5a58003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a67fff700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a58004850 0x7f5a58004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.412+0000 7f5a6d2b7700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 0x7f5a58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6cab6700 1 -- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 msgr2=0x7f5a58005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a680b06c0 0x7f5a58005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6cab6700 1 -- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a58004850 msgr2=0x7f5a58004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:04.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a58004850 0x7f5a58004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6cab6700 1 -- 172.21.15.79:0/1485078343 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a580035b0 con 0x7f5a58003a70 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a58003a70 0x7f5a58003e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009b90 tx=0x7f5a58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc72db9cff9844ff server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00e6c0 con 0x7f5a58003a70 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00e820 con 0x7f5a58003a70 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a677fe700 1 -- 172.21.15.79:0/1485078343 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00eb90 con 0x7f5a58003a70 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/1485078343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a58003a70 msgr2=0x7f5a58003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/1485078343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a58003a70 0x7f5a58003e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009b90 tx=0x7f5a58005880 comp rx=0 tx=0).stop 2023-06-08T03:51:04.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6d2b7700 1 -- 172.21.15.79:0/1485078343 reap_dead start 2023-06-08T03:51:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/1485078343 shutdown_connections 2023-06-08T03:51:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/1485078343 >> 172.21.15.79:0/1485078343 conn(0x7f5a68069d50 msgr2=0x7f5a6806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/1485078343 shutdown_connections 2023-06-08T03:51:04.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.413+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/1485078343 wait complete. 2023-06-08T03:51:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.415+0000 7f5a6ed1a700 1 Processor -- start 2023-06-08T03:51:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.415+0000 7f5a6ed1a700 1 -- start start 2023-06-08T03:51:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.415+0000 7f5a6ed1a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a58003a70 0x7f5a68150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.415+0000 7f5a6ed1a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a681e5e40 0x7f5a680ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6ed1a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 0x7f5a681e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6ed1a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68150970 con 0x7f5a681e62b0 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6ed1a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681e4de0 con 0x7f5a681e5e40 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6ed1a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a681e5080 con 0x7f5a58003a70 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 0x7f5a681e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 0x7f5a681e23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35750/0 (socket says 172.21.15.79:35750) 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 -- 172.21.15.79:0/673771138 learned_addr learned my addr 172.21.15.79:0/673771138 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a58003a70 0x7f5a68150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a67fff700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a681e5e40 0x7f5a680ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 -- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a58003a70 msgr2=0x7f5a68150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a58003a70 0x7f5a68150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 -- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a681e5e40 msgr2=0x7f5a680ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a681e5e40 0x7f5a680ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 -- 172.21.15.79:0/673771138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a681e5360 con 0x7f5a681e62b0 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6d2b7700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 0x7f5a681e23e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a680b45e0 tx=0x7f5a60011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:04.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000e040 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000f040 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a681e5640 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000fc40 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.416+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a68150c70 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.417+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a681e5b60 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.418+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a60022460 con 0x7f5a681e62b0 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.418+0000 7f5a657fa700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a4c06ace0 0x7f5a4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.418+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a60089be0 con 0x7f5a681e62b0 2023-06-08T03:51:04.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.418+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a4c06ace0 0x7f5a4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.419+0000 7f5a6cab6700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a4c06ace0 0x7f5a4c06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0090b0 tx=0x7f5a5c01f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:04.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.422+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a681e5b60 con 0x7f5a681e62b0 2023-06-08T03:51:04.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.609+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a681e5b60 con 0x7f5a681e62b0 2023-06-08T03:51:04.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.610+0000 7f5a657fa700 1 -- 172.21.15.79:0/673771138 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f5a681e5b60 con 0x7f5a681e62b0 2023-06-08T03:51:04.614 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:04.614 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:04.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a4c06ace0 msgr2=0x7f5a4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a4c06ace0 0x7f5a4c06d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0090b0 tx=0x7f5a5c01f040 comp rx=0 tx=0).stop 2023-06-08T03:51:04.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 msgr2=0x7f5a681e23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 0x7f5a681e23e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a680b45e0 tx=0x7f5a60011ed0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 shutdown_connections 2023-06-08T03:51:04.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a681e62b0 0x7f5a681e23e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a4c06ace0 0x7f5a4c06d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a58003a70 0x7f5a68150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.612+0000 7f5a6ed1a700 1 --2- 172.21.15.79:0/673771138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a681e5e40 0x7f5a680ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.613+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 >> 172.21.15.79:0/673771138 conn(0x7f5a68069d50 msgr2=0x7f5a680b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:04.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.613+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 shutdown_connections 2023-06-08T03:51:04.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.613+0000 7f5a6ed1a700 1 -- 172.21.15.79:0/673771138 wait complete. 2023-06-08T03:51:04.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:04.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.751+0000 7f7b47017700 1 Processor -- start 2023-06-08T03:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.751+0000 7f7b47017700 1 -- start start 2023-06-08T03:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b47017700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b400d0240 0x7f7b400af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b47017700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 0x7f7b400cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b47017700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4015d930 con 0x7f7b400b20f0 2023-06-08T03:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b47017700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4015e770 con 0x7f7b400d0240 2023-06-08T03:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b47017700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b400cd9b0 con 0x7f7b400cd000 2023-06-08T03:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 0x7f7b400cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 0x7f7b400cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:50076/0 (socket says 172.21.15.79:50076) 2023-06-08T03:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 -- 172.21.15.79:0/4130545874 learned_addr learned my addr 172.21.15.79:0/4130545874 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b44db3700 1 --2- 172.21.15.79:0/4130545874 >> v2:172.21.15.156:3300/0 conn(0x7f7b400d0240 0x7f7b400af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 -- 172.21.15.79:0/4130545874 >> v2:172.21.15.156:3300/0 conn(0x7f7b400d0240 msgr2=0x7f7b400af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 --2- 172.21.15.79:0/4130545874 >> v2:172.21.15.156:3300/0 conn(0x7f7b400d0240 0x7f7b400af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 -- 172.21.15.79:0/4130545874 >> v1:172.21.15.156:6789/0 conn(0x7f7b400b20f0 legacy=0x7f7b400b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.752+0000 7f7b3ffff700 1 -- 172.21.15.79:0/4130545874 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4015db60 con 0x7f7b400cd000 2023-06-08T03:51:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b3ffff700 1 --2- 172.21.15.79:0/4130545874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 0x7f7b400cd470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0098f0 tx=0x7f7b2c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d3cbd1a688ec53 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b3f7fe700 1 -- 172.21.15.79:0/4130545874 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011830 con 0x7f7b400cd000 2023-06-08T03:51:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b3f7fe700 1 -- 172.21.15.79:0/4130545874 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c011990 con 0x7f7b400cd000 2023-06-08T03:51:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b3f7fe700 1 -- 172.21.15.79:0/4130545874 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011d20 con 0x7f7b400cd000 2023-06-08T03:51:04.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 -- 172.21.15.79:0/4130545874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 msgr2=0x7f7b400cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 --2- 172.21.15.79:0/4130545874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 0x7f7b400cd470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0098f0 tx=0x7f7b2c009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 -- 172.21.15.79:0/4130545874 shutdown_connections 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 --2- 172.21.15.79:0/4130545874 >> v2:172.21.15.156:3300/0 conn(0x7f7b400d0240 0x7f7b400af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 --2- 172.21.15.79:0/4130545874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400cd000 0x7f7b400cd470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 -- 172.21.15.79:0/4130545874 >> 172.21.15.79:0/4130545874 conn(0x7f7b4006e1e0 msgr2=0x7f7b4006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 -- 172.21.15.79:0/4130545874 shutdown_connections 2023-06-08T03:51:04.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.753+0000 7f7b47017700 1 -- 172.21.15.79:0/4130545874 wait complete. 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.754+0000 7f7b47017700 1 Processor -- start 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 -- start start 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400b20f0 0x7f7b400b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 0x7f7b4015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400d0240 0x7f7b4014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4015c9a0 con 0x7f7b400cd000 2023-06-08T03:51:04.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4014e9f0 con 0x7f7b400d0240 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.755+0000 7f7b47017700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b400c9da0 con 0x7f7b400b20f0 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 0x7f7b4015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 0x7f7b4015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35764/0 (socket says 172.21.15.79:35764) 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 -- 172.21.15.79:0/3647112616 learned_addr learned my addr 172.21.15.79:0/3647112616 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b44db3700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400b20f0 0x7f7b400b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b455b4700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400d0240 0x7f7b4014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 -- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400b20f0 msgr2=0x7f7b400b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400b20f0 0x7f7b400b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 -- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400d0240 msgr2=0x7f7b4014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400d0240 0x7f7b4014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 -- 172.21.15.79:0/3647112616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c009550 con 0x7f7b400cd000 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3ffff700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 0x7f7b4015ce30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009c40 tx=0x7f7b2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c004400 con 0x7f7b400cd000 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c01f070 con 0x7f7b400cd000 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b400ca080 con 0x7f7b400cd000 2023-06-08T03:51:04.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c0038e0 con 0x7f7b400cd000 2023-06-08T03:51:04.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.756+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4015c630 con 0x7f7b400cd000 2023-06-08T03:51:04.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.757+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b400ca290 con 0x7f7b400cd000 2023-06-08T03:51:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.757+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b2c021030 con 0x7f7b400cd000 2023-06-08T03:51:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.758+0000 7f7b3d7fa700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7b2406ad10 0x7f7b2406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.758+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b2c08abe0 con 0x7f7b400cd000 2023-06-08T03:51:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.758+0000 7f7b44db3700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7b2406ad10 0x7f7b2406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.758+0000 7f7b44db3700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7b2406ad10 0x7f7b2406d1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009be0 tx=0x7f7b34009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:04.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.762+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b2c055b50 con 0x7f7b400cd000 2023-06-08T03:51:04.954 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.951+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7b40073e90 con 0x7f7b400cd000 2023-06-08T03:51:04.955 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.951+0000 7f7b3d7fa700 1 -- 172.21.15.79:0/3647112616 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7b2c0150a0 con 0x7f7b400cd000 2023-06-08T03:51:04.955 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:04.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7b2406ad10 msgr2=0x7f7b2406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7b2406ad10 0x7f7b2406d1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009be0 tx=0x7f7b34009210 comp rx=0 tx=0).stop 2023-06-08T03:51:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 msgr2=0x7f7b4015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 0x7f7b4015ce30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009c40 tx=0x7f7b2c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 shutdown_connections 2023-06-08T03:51:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400d0240 0x7f7b4014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7b2406ad10 0x7f7b2406d1c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b400b20f0 0x7f7b400b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 --2- 172.21.15.79:0/3647112616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7b400cd000 0x7f7b4015ce30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 >> 172.21.15.79:0/3647112616 conn(0x7f7b4006e1e0 msgr2=0x7f7b400b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 shutdown_connections 2023-06-08T03:51:04.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:04.953+0000 7f7b47017700 1 -- 172.21.15.79:0/3647112616 wait complete. 2023-06-08T03:51:04.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:05.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.087+0000 7f04405c1700 1 Processor -- start 2023-06-08T03:51:05.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.087+0000 7f04405c1700 1 -- start start 2023-06-08T03:51:05.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f04405c1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f04380cc690 0x7f0438163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f04405c1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cd7b0 con 0x7f04380ca360 2023-06-08T03:51:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f04405c1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380cda50 con 0x7f0438158d50 2023-06-08T03:51:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f04405c1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380cdd30 con 0x7f04380cc690 2023-06-08T03:51:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f043db5c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f04380ca360 0x7f04380cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59244/0 (socket says 172.21.15.79:59244) 2023-06-08T03:51:05.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f043eb5e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f04380cc690 0x7f0438163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.088+0000 7f043db5c700 1 -- 172.21.15.79:0/9381608 learned_addr learned my addr 172.21.15.79:0/9381608 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043d35b700 1 -- 172.21.15.79:0/9381608 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972174132 0 0) 0x7f04380cd7b0 con 0x7f04380ca360 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043d35b700 1 -- 172.21.15.79:0/9381608 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04240035b0 con 0x7f04380ca360 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043eb5e700 1 -- 172.21.15.79:0/9381608 >> v1:172.21.15.156:6789/0 conn(0x7f0438158d50 legacy=0x7f043815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043eb5e700 1 -- 172.21.15.79:0/9381608 >> v1:172.21.15.79:6789/0 conn(0x7f04380ca360 legacy=0x7f04380cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043eb5e700 1 -- 172.21.15.79:0/9381608 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380ce010 con 0x7f04380cc690 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043eb5e700 1 --2- 172.21.15.79:0/9381608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380cc690 0x7f0438163d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0430009d90 tx=0x7f043000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf56d1921c862d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043d35b700 1 -- 172.21.15.79:0/9381608 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000e070 con 0x7f04380cc690 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043d35b700 1 -- 172.21.15.79:0/9381608 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043000bb20 con 0x7f04380cc690 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f043d35b700 1 -- 172.21.15.79:0/9381608 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430003a20 con 0x7f04380cc690 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f04405c1700 1 -- 172.21.15.79:0/9381608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380cc690 msgr2=0x7f0438163d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.089+0000 7f04405c1700 1 --2- 172.21.15.79:0/9381608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380cc690 0x7f0438163d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0430009d90 tx=0x7f043000b330 comp rx=0 tx=0).stop 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.090+0000 7f04405c1700 1 -- 172.21.15.79:0/9381608 shutdown_connections 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.090+0000 7f04405c1700 1 --2- 172.21.15.79:0/9381608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380cc690 0x7f0438163d50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.090+0000 7f04405c1700 1 -- 172.21.15.79:0/9381608 >> 172.21.15.79:0/9381608 conn(0x7f04380b1d30 msgr2=0x7f043806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.090+0000 7f04405c1700 1 -- 172.21.15.79:0/9381608 shutdown_connections 2023-06-08T03:51:05.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.090+0000 7f04405c1700 1 -- 172.21.15.79:0/9381608 wait complete. 2023-06-08T03:51:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.091+0000 7f04405c1700 1 Processor -- start 2023-06-08T03:51:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.091+0000 7f04405c1700 1 -- start start 2023-06-08T03:51:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 0x7f04380c2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04380cc690 0x7f04380adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.098 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0438158d50 0x7f043814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c3e80 con 0x7f04380ca360 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380ae100 con 0x7f04380cc690 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380c3270 con 0x7f0438158d50 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 0x7f04380c2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 0x7f04380c2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35774/0 (socket says 172.21.15.79:35774) 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 -- 172.21.15.79:0/2962441413 learned_addr learned my addr 172.21.15.79:0/2962441413 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:05.099 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043db5c700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04380cc690 0x7f04380adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043eb5e700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0438158d50 0x7f043814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 -- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0438158d50 msgr2=0x7f043814d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0438158d50 0x7f043814d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 -- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04380cc690 msgr2=0x7f04380adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04380cc690 0x7f04380adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 -- 172.21.15.79:0/2962441413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430009a70 con 0x7f04380ca360 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f043e35d700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 0x7f04380c2c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f042800b4a0 tx=0x7f042800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.100 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428011830 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0428011990 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04380c3550 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0428011d40 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.092+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04380c3b10 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.093+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04380708d0 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.094+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f042801a400 con 0x7f04380ca360 2023-06-08T03:51:05.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.094+0000 7f042effd700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f041c06ace0 0x7f041c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.094+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0428089460 con 0x7f04380ca360 2023-06-08T03:51:05.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.094+0000 7f043db5c700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f041c06ace0 0x7f041c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.095+0000 7f043db5c700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f041c06ace0 0x7f041c06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04380cc2f0 tx=0x7f0434009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.098+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f04280542b0 con 0x7f04380ca360 2023-06-08T03:51:05.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.284+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0438073e90 con 0x7f04380ca360 2023-06-08T03:51:05.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.284+0000 7f042effd700 1 -- 172.21.15.79:0/2962441413 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04280578d0 con 0x7f04380ca360 2023-06-08T03:51:05.288 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:05.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f041c06ace0 msgr2=0x7f041c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f041c06ace0 0x7f041c06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f04380cc2f0 tx=0x7f0434009040 comp rx=0 tx=0).stop 2023-06-08T03:51:05.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 msgr2=0x7f04380c2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 0x7f04380c2c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f042800b4a0 tx=0x7f042800b860 comp rx=0 tx=0).stop 2023-06-08T03:51:05.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 shutdown_connections 2023-06-08T03:51:05.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f041c06ace0 0x7f041c06d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0438158d50 0x7f043814d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f04380ca360 0x7f04380c2c90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.286+0000 7f04405c1700 1 --2- 172.21.15.79:0/2962441413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04380cc690 0x7f04380adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.287+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 >> 172.21.15.79:0/2962441413 conn(0x7f04380b1d30 msgr2=0x7f0438071120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:05.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.287+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 shutdown_connections 2023-06-08T03:51:05.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.287+0000 7f04405c1700 1 -- 172.21.15.79:0/2962441413 wait complete. 2023-06-08T03:51:05.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:05.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.423+0000 7f6face35700 1 Processor -- start 2023-06-08T03:51:05.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.423+0000 7f6face35700 1 -- start start 2023-06-08T03:51:05.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.424+0000 7f6face35700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6fa806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.424+0000 7f6face35700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa80cd140 con 0x7f6fa8069d50 2023-06-08T03:51:05.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.424+0000 7f6face35700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa8071110 con 0x7f6fa8071fe0 2023-06-08T03:51:05.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.424+0000 7f6face35700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8071be0 con 0x7f6fa815c1f0 2023-06-08T03:51:05.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa6d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6fa8069d50 0x7f6fa80cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59262/0 (socket says 172.21.15.79:59262) 2023-06-08T03:51:05.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa6d9d700 1 -- 172.21.15.79:0/3041883211 learned_addr learned my addr 172.21.15.79:0/3041883211 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:05.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa659c700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6fa806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807125702 0 0) 0x7f6fa80cd140 con 0x7f6fa8069d50 2023-06-08T03:51:05.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f900035b0 con 0x7f6fa8069d50 2023-06-08T03:51:05.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996613342 0 0) 0x7f6fa8071110 con 0x7f6fa8071fe0 2023-06-08T03:51:05.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa80cd140 con 0x7f6fa8071fe0 2023-06-08T03:51:05.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2819198491 0 0) 0x7f6f900035b0 con 0x7f6fa8069d50 2023-06-08T03:51:05.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 msgr2=0x7f6fa806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6fa806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 >> v1:172.21.15.156:6789/0 conn(0x7f6fa8071fe0 legacy=0x7f6fa80cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80cd390 con 0x7f6fa8069d50 2023-06-08T03:51:05.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f6fa00019f0 con 0x7f6fa8069d50 2023-06-08T03:51:05.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa659c700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6fa806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:05.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 >> v1:172.21.15.79:6789/0 conn(0x7f6fa8069d50 legacy=0x7f6fa80cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90003bc0 0x7f6f90003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f90004850 0x7f6f90004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6f90005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8071be0 con 0x7f6f90004850 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9000bba0 con 0x7f6f90003bc0 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9000bd30 con 0x7f6fa815c1f0 2023-06-08T03:51:05.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa5d9b700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f90004850 0x7f6f90004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa659c700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90003bc0 0x7f6f90003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.425+0000 7f6fa6d9d700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6f90005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa5d9b700 1 -- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 msgr2=0x7f6f90005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa5d9b700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa815c1f0 0x7f6f90005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa5d9b700 1 -- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90003bc0 msgr2=0x7f6f90003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa5d9b700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90003bc0 0x7f6f90003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa5d9b700 1 -- 172.21.15.79:0/3041883211 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0094a0 con 0x7f6f90004850 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa5d9b700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f90004850 0x7f6f90004cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f98001940 tx=0x7f6f90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f3c2510ae65807 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f980052d0 con 0x7f6f90004850 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98001d70 con 0x7f6f90004850 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa559a700 1 -- 172.21.15.79:0/3041883211 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98003680 con 0x7f6f90004850 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6face35700 1 -- 172.21.15.79:0/3041883211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f90004850 msgr2=0x7f6f90004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6face35700 1 --2- 172.21.15.79:0/3041883211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f90004850 0x7f6f90004cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f98001940 tx=0x7f6f90005880 comp rx=0 tx=0).stop 2023-06-08T03:51:05.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6fa6d9d700 1 -- 172.21.15.79:0/3041883211 reap_dead start 2023-06-08T03:51:05.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6face35700 1 -- 172.21.15.79:0/3041883211 shutdown_connections 2023-06-08T03:51:05.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6face35700 1 -- 172.21.15.79:0/3041883211 >> 172.21.15.79:0/3041883211 conn(0x7f6fa80b1d30 msgr2=0x7f6fa806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:05.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.426+0000 7f6face35700 1 -- 172.21.15.79:0/3041883211 shutdown_connections 2023-06-08T03:51:05.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.427+0000 7f6face35700 1 -- 172.21.15.79:0/3041883211 wait complete. 2023-06-08T03:51:05.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.428+0000 7f6face35700 1 Processor -- start 2023-06-08T03:51:05.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.428+0000 7f6face35700 1 -- start start 2023-06-08T03:51:05.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90004850 0x7f6fa81f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa8152550 0x7f6fa8151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 0x7f6fa80ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa81f6130 con 0x7f6fa8156770 2023-06-08T03:51:05.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8152390 con 0x7f6f90004850 2023-06-08T03:51:05.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa81f5890 con 0x7f6fa8152550 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 0x7f6fa80ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 0x7f6fa80ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35798/0 (socket says 172.21.15.79:35798) 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 -- 172.21.15.79:0/3516788140 learned_addr learned my addr 172.21.15.79:0/3516788140 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa659c700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90004850 0x7f6fa81f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa5d9b700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa8152550 0x7f6fa8151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 -- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa8152550 msgr2=0x7f6fa8151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa8152550 0x7f6fa8151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 -- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90004850 msgr2=0x7f6fa81f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90004850 0x7f6fa81f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 -- 172.21.15.79:0/3516788140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa81f5b70 con 0x7f6fa8156770 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6fa6d9d700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 0x7f6fa80ca010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80cba90 tx=0x7f6fa000a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0001410 con 0x7f6fa8156770 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa000ace0 con 0x7f6fa8156770 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa81f5e50 con 0x7f6fa8156770 2023-06-08T03:51:05.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0013630 con 0x7f6fa8156770 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.429+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa8152d40 con 0x7f6fa8156770 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.430+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa80c6a20 con 0x7f6fa8156770 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.431+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6fa0013790 con 0x7f6fa8156770 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.431+0000 7f6f96ffd700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6f8806ad10 0x7f6f8806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.431+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6fa008a060 con 0x7f6fa8156770 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.431+0000 7f6fa659c700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6f8806ad10 0x7f6f8806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.432+0000 7f6fa659c700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6f8806ad10 0x7f6f8806d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009c50 tx=0x7f6f9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.435+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6fa80c6a20 con 0x7f6fa8156770 2023-06-08T03:51:05.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.621+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6fa80c6a20 con 0x7f6fa8156770 2023-06-08T03:51:05.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.622+0000 7f6f96ffd700 1 -- 172.21.15.79:0/3516788140 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fa80c6a20 con 0x7f6fa8156770 2023-06-08T03:51:05.625 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:05.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6f8806ad10 msgr2=0x7f6f8806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6f8806ad10 0x7f6f8806d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009c50 tx=0x7f6f9c008040 comp rx=0 tx=0).stop 2023-06-08T03:51:05.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 msgr2=0x7f6fa80ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 0x7f6fa80ca010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80cba90 tx=0x7f6fa000a5b0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 shutdown_connections 2023-06-08T03:51:05.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fa8156770 0x7f6fa80ca010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f90004850 0x7f6fa81f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6f8806ad10 0x7f6f8806d1c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.623+0000 7f6face35700 1 --2- 172.21.15.79:0/3516788140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fa8152550 0x7f6fa8151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.624+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 >> 172.21.15.79:0/3516788140 conn(0x7f6fa80b1d30 msgr2=0x7f6fa80b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:05.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.624+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 shutdown_connections 2023-06-08T03:51:05.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.624+0000 7f6face35700 1 -- 172.21.15.79:0/3516788140 wait complete. 2023-06-08T03:51:05.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:05.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.758+0000 7f30acbc3700 1 Processor -- start 2023-06-08T03:51:05.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.758+0000 7f30acbc3700 1 -- start start 2023-06-08T03:51:05.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.759+0000 7f30acbc3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f30a80707d0 0x7f30a8070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.759+0000 7f30acbc3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a8071290 con 0x7f30a80cacc0 2023-06-08T03:51:05.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.759+0000 7f30acbc3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a8071570 con 0x7f30a815a0e0 2023-06-08T03:51:05.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.759+0000 7f30acbc3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a80ca4c0 con 0x7f30a80707d0 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.759+0000 7f30a6d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f30a80cacc0 0x7f30a806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59282/0 (socket says 172.21.15.79:59282) 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.759+0000 7f30a6d9d700 1 -- 172.21.15.79:0/1740418705 learned_addr learned my addr 172.21.15.79:0/1740418705 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a5d9b700 1 --2- 172.21.15.79:0/1740418705 >> v2:172.21.15.156:3300/0 conn(0x7f30a80707d0 0x7f30a8070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399015248 0 0) 0x7f30a8071290 con 0x7f30a80cacc0 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30900035b0 con 0x7f30a80cacc0 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2345643192 0 0) 0x7f30900035b0 con 0x7f30a80cacc0 2023-06-08T03:51:05.766 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 >> v2:172.21.15.156:3300/0 conn(0x7f30a80707d0 msgr2=0x7f30a8070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 --2- 172.21.15.79:0/1740418705 >> v2:172.21.15.156:3300/0 conn(0x7f30a80707d0 0x7f30a8070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 >> v1:172.21.15.156:6789/0 conn(0x7f30a815a0e0 legacy=0x7f30a8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a80ca7a0 con 0x7f30a80cacc0 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f30a00019f0 con 0x7f30a80cacc0 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 >> v1:172.21.15.79:6789/0 conn(0x7f30a80cacc0 legacy=0x7f30a806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3090003a00 0x7f3090003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3090004850 0x7f3090004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.767 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090005200 0x7f309000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80ca4c0 con 0x7f3090003a00 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309000c110 con 0x7f3090005200 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309000c2d0 con 0x7f3090004850 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3090003a00 0x7f3090003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a6d9d700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090005200 0x7f309000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a5d9b700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3090004850 0x7f3090004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 -- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3090004850 msgr2=0x7f3090004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3090004850 0x7f3090004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 -- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090005200 msgr2=0x7f309000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090005200 0x7f309000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 -- 172.21.15.79:0/1740418705 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30900035b0 con 0x7f3090003a00 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a6d9d700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090005200 0x7f309000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a6d9d700 1 -- 172.21.15.79:0/1740418705 reap_dead start 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.760+0000 7f30a659c700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3090003a00 0x7f3090003e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f309c0017c0 tx=0x7f3090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fcbf9dff1ef9d0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0052d0 con 0x7f3090003a00 2023-06-08T03:51:05.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c005430 con 0x7f3090003a00 2023-06-08T03:51:05.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30a559a700 1 -- 172.21.15.79:0/1740418705 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0038c0 con 0x7f3090003a00 2023-06-08T03:51:05.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 -- 172.21.15.79:0/1740418705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3090003a00 msgr2=0x7f3090003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3090003a00 0x7f3090003e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f309c0017c0 tx=0x7f3090005880 comp rx=0 tx=0).stop 2023-06-08T03:51:05.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 -- 172.21.15.79:0/1740418705 shutdown_connections 2023-06-08T03:51:05.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 --2- 172.21.15.79:0/1740418705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3090003a00 0x7f3090003e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 -- 172.21.15.79:0/1740418705 >> 172.21.15.79:0/1740418705 conn(0x7f30a80ced30 msgr2=0x7f30a80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:05.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 -- 172.21.15.79:0/1740418705 shutdown_connections 2023-06-08T03:51:05.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.761+0000 7f30acbc3700 1 -- 172.21.15.79:0/1740418705 wait complete. 2023-06-08T03:51:05.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 Processor -- start 2023-06-08T03:51:05.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 -- start start 2023-06-08T03:51:05.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090003a00 0x7f30a80c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 0x7f30a81f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a81f1be0 0x7f30a80c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b2cc0 con 0x7f30a80c2860 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b2fa0 con 0x7f3090003a00 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30acbc3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b3280 con 0x7f30a81f1be0 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30a5d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 0x7f30a81f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30a5d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 0x7f30a81f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35814/0 (socket says 172.21.15.79:35814) 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.763+0000 7f30a5d9b700 1 -- 172.21.15.79:0/3270442144 learned_addr learned my addr 172.21.15.79:0/3270442144 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a659c700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090003a00 0x7f30a80c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a6d9d700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a81f1be0 0x7f30a80c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a5d9b700 1 -- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a81f1be0 msgr2=0x7f30a80c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a5d9b700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a81f1be0 0x7f30a80c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a5d9b700 1 -- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090003a00 msgr2=0x7f30a80c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a5d9b700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090003a00 0x7f30a80c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a5d9b700 1 -- 172.21.15.79:0/3270442144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a80b3560 con 0x7f30a80c2860 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30a5d9b700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 0x7f30a81f16a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30980116f0 tx=0x7f3098011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098015870 con 0x7f30a80c2860 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30980159d0 con 0x7f30a80c2860 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098015d40 con 0x7f30a80c2860 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a80c6b30 con 0x7f30a80c2860 2023-06-08T03:51:05.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.764+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a80c88d0 con 0x7f30a80c2860 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.765+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a80b38d0 con 0x7f30a80c2860 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.765+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3098013400 con 0x7f30a80c2860 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.766+0000 7f3096ffd700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308806ace0 0x7f308806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.766+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3098089b10 con 0x7f30a80c2860 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.766+0000 7f30a659c700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308806ace0 0x7f308806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.766+0000 7f30a659c700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308806ace0 0x7f308806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f309c000c00 tx=0x7f309c009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:05.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.770+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f30980548d0 con 0x7f30a80c2860 2023-06-08T03:51:05.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.962+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30a80b2e00 con 0x7f30a80c2860 2023-06-08T03:51:05.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.962+0000 7f3096ffd700 1 -- 172.21.15.79:0/3270442144 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30a80b2e00 con 0x7f30a80c2860 2023-06-08T03:51:05.965 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:05.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308806ace0 msgr2=0x7f308806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308806ace0 0x7f308806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f309c000c00 tx=0x7f309c009030 comp rx=0 tx=0).stop 2023-06-08T03:51:05.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 msgr2=0x7f30a81f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:05.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 0x7f30a81f16a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f30980116f0 tx=0x7f3098011ab0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 shutdown_connections 2023-06-08T03:51:05.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f308806ace0 0x7f308806d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30a81f1be0 0x7f30a80c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3090003a00 0x7f30a80c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 --2- 172.21.15.79:0/3270442144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30a80c2860 0x7f30a81f16a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:05.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 >> 172.21.15.79:0/3270442144 conn(0x7f30a80ced30 msgr2=0x7f30a80afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:05.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.964+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 shutdown_connections 2023-06-08T03:51:05.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:05.965+0000 7f30acbc3700 1 -- 172.21.15.79:0/3270442144 wait complete. 2023-06-08T03:51:05.979 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:06.101 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 Processor -- start 2023-06-08T03:51:06.102 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 -- start start 2023-06-08T03:51:06.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 0x7f328c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f328c06d6a0 0x7f328c070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c06db00 con 0x7f328c158ed0 2023-06-08T03:51:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328c0cd700 con 0x7f328c06d6a0 2023-06-08T03:51:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.099+0000 7f32924bd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1592a0 con 0x7f328c0cd290 2023-06-08T03:51:06.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328bfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f328c158ed0 0x7f328c0c9900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59298/0 (socket says 172.21.15.79:59298) 2023-06-08T03:51:06.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328bfff700 1 -- 172.21.15.79:0/3058842531 learned_addr learned my addr 172.21.15.79:0/3058842531 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:06.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224802149 0 0) 0x7f328c06db00 con 0x7f328c158ed0 2023-06-08T03:51:06.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32740035b0 con 0x7f328c158ed0 2023-06-08T03:51:06.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 --2- 172.21.15.79:0/3058842531 >> v2:172.21.15.156:3300/0 conn(0x7f328c06d6a0 0x7f328c070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328b7fe700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 0x7f328c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436712213 0 0) 0x7f32740035b0 con 0x7f328c158ed0 2023-06-08T03:51:06.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 msgr2=0x7f328c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 0x7f328c06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 >> v2:172.21.15.156:3300/0 conn(0x7f328c06d6a0 msgr2=0x7f328c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 --2- 172.21.15.79:0/3058842531 >> v2:172.21.15.156:3300/0 conn(0x7f328c06d6a0 0x7f328c070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0cd990 con 0x7f328c158ed0 2023-06-08T03:51:06.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f327c001670 con 0x7f328c158ed0 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 >> v1:172.21.15.79:6789/0 conn(0x7f328c158ed0 legacy=0x7f328c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 0x7f3274003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274003f80 0x7f3274004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3274004d90 0x7f3274005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0cd700 con 0x7f3274004d90 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1592a0 con 0x7f3274003f80 2023-06-08T03:51:06.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327400dc20 con 0x7f328c0cd290 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3274004d90 0x7f3274005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328bfff700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 0x7f3274003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f328b7fe700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274003f80 0x7f3274004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 -- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 msgr2=0x7f3274003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0cd290 0x7f3274003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 -- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274003f80 msgr2=0x7f3274004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274003f80 0x7f3274004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.100+0000 7f3290a5a700 1 -- 172.21.15.79:0/3058842531 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32740035b0 con 0x7f3274004d90 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f3290a5a700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3274004d90 0x7f3274005200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3284009fc0 tx=0x7f3274005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdc455caf2e2755 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328400ec30 con 0x7f3274004d90 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328400f040 con 0x7f3274004d90 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f328affd700 1 -- 172.21.15.79:0/3058842531 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284014480 con 0x7f3274004d90 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f32924bd700 1 -- 172.21.15.79:0/3058842531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3274004d90 msgr2=0x7f3274005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f32924bd700 1 --2- 172.21.15.79:0/3058842531 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3274004d90 0x7f3274005200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3284009fc0 tx=0x7f3274005880 comp rx=0 tx=0).stop 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f3290a5a700 1 -- 172.21.15.79:0/3058842531 reap_dead start 2023-06-08T03:51:06.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f32924bd700 1 -- 172.21.15.79:0/3058842531 shutdown_connections 2023-06-08T03:51:06.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f32924bd700 1 -- 172.21.15.79:0/3058842531 >> 172.21.15.79:0/3058842531 conn(0x7f328c0ce4e0 msgr2=0x7f328c0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:06.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f32924bd700 1 -- 172.21.15.79:0/3058842531 shutdown_connections 2023-06-08T03:51:06.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.101+0000 7f32924bd700 1 -- 172.21.15.79:0/3058842531 wait complete. 2023-06-08T03:51:06.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 Processor -- start 2023-06-08T03:51:06.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 -- start start 2023-06-08T03:51:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 0x7f328c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f328c15b970 0x7f328c06c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0b17d0 0x7f328c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c6360 con 0x7f328c1ece50 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c56e0 con 0x7f328c15b970 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f32924bd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c0c59c0 con 0x7f328c0b17d0 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.103+0000 7f328bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 0x7f328c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 0x7f328c159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35834/0 (socket says 172.21.15.79:35834) 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 -- 172.21.15.79:0/2341703700 learned_addr learned my addr 172.21.15.79:0/2341703700 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:06.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f3290a5a700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0b17d0 0x7f328c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328b7fe700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f328c15b970 0x7f328c06c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 -- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0b17d0 msgr2=0x7f328c06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0b17d0 0x7f328c06c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 -- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f328c15b970 msgr2=0x7f328c06c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f328c15b970 0x7f328c06c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 -- 172.21.15.79:0/2341703700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0c5ca0 con 0x7f328c1ece50 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328b7fe700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f328c15b970 0x7f328c06c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:06.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f328bfff700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 0x7f328c159e80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f327c001cc0 tx=0x7f327c00f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c007d30 con 0x7f328c1ece50 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c00fa90 con 0x7f328c1ece50 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c00fe40 con 0x7f328c1ece50 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c0c5f80 con 0x7f328c1ece50 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.104+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c1ed220 con 0x7f328c1ece50 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.105+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c073e90 con 0x7f328c1ece50 2023-06-08T03:51:06.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.105+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f327c01d460 con 0x7f328c1ece50 2023-06-08T03:51:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.106+0000 7f3288ff9700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f327806ace0 0x7f327806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.106+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f327c089900 con 0x7f328c1ece50 2023-06-08T03:51:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.106+0000 7f328b7fe700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f327806ace0 0x7f327806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.107+0000 7f328b7fe700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f327806ace0 0x7f327806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f328c1f0b50 tx=0x7f3280008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.110+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f327c054750 con 0x7f328c1ece50 2023-06-08T03:51:06.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.296+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f328c0ad300 con 0x7f328c1ece50 2023-06-08T03:51:06.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.296+0000 7f3288ff9700 1 -- 172.21.15.79:0/2341703700 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f327c057d70 con 0x7f328c1ece50 2023-06-08T03:51:06.300 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:06.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f327806ace0 msgr2=0x7f327806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f327806ace0 0x7f327806d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f328c1f0b50 tx=0x7f3280008040 comp rx=0 tx=0).stop 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 msgr2=0x7f328c159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 0x7f328c159e80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f327c001cc0 tx=0x7f327c00f4b0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 shutdown_connections 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f327806ace0 0x7f327806d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f328c0b17d0 0x7f328c06c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f328c1ece50 0x7f328c159e80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 --2- 172.21.15.79:0/2341703700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f328c15b970 0x7f328c06c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 >> 172.21.15.79:0/2341703700 conn(0x7f328c0ce4e0 msgr2=0x7f328c070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:06.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.298+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 shutdown_connections 2023-06-08T03:51:06.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.299+0000 7f32924bd700 1 -- 172.21.15.79:0/2341703700 wait complete. 2023-06-08T03:51:06.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:06.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.429+0000 7f3a71134700 1 Processor -- start 2023-06-08T03:51:06.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.429+0000 7f3a71134700 1 -- start start 2023-06-08T03:51:06.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a71134700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a6c15a640 0x7f3a6c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a71134700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c15aa10 0x7f3a6c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a71134700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3a6c0ce930 0x7f3a6c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a71134700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c0b0750 con 0x7f3a6c15a640 2023-06-08T03:51:06.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a71134700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ceeb0 con 0x7f3a6c0ce930 2023-06-08T03:51:06.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a71134700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0b09a0 con 0x7f3a6c15aa10 2023-06-08T03:51:06.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a6c15a640 0x7f3a6c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3a6c15a640 0x7f3a6c0b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35842/0 (socket says 172.21.15.79:35842) 2023-06-08T03:51:06.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 -- 172.21.15.79:0/519021625 learned_addr learned my addr 172.21.15.79:0/519021625 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:06.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6a59c700 1 --2- 172.21.15.79:0/519021625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c15aa10 0x7f3a6c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6b59e700 1 --2- 172.21.15.79:0/519021625 >> v2:172.21.15.156:3300/0 conn(0x7f3a6c0ce930 0x7f3a6c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 -- 172.21.15.79:0/519021625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c15aa10 msgr2=0x7f3a6c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 --2- 172.21.15.79:0/519021625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c15aa10 0x7f3a6c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 -- 172.21.15.79:0/519021625 >> v2:172.21.15.156:3300/0 conn(0x7f3a6c0ce930 msgr2=0x7f3a6c0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 --2- 172.21.15.79:0/519021625 >> v2:172.21.15.156:3300/0 conn(0x7f3a6c0ce930 0x7f3a6c0af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 -- 172.21.15.79:0/519021625 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0b0c80 con 0x7f3a6c15a640 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a6ad9d700 1 --2- 172.21.15.79:0/519021625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0b1480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0098f0 tx=0x7f3a5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b0342e7da333b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a69d9b700 1 -- 172.21.15.79:0/519021625 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00b860 con 0x7f3a6c15a640 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a69d9b700 1 -- 172.21.15.79:0/519021625 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5c00b9c0 con 0x7f3a6c15a640 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.430+0000 7f3a69d9b700 1 -- 172.21.15.79:0/519021625 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c00bd50 con 0x7f3a6c15a640 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 -- 172.21.15.79:0/519021625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 msgr2=0x7f3a6c0b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 --2- 172.21.15.79:0/519021625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0b1480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0098f0 tx=0x7f3a5c009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 -- 172.21.15.79:0/519021625 shutdown_connections 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 --2- 172.21.15.79:0/519021625 >> v2:172.21.15.156:3300/0 conn(0x7f3a6c0ce930 0x7f3a6c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 --2- 172.21.15.79:0/519021625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0b1480 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 --2- 172.21.15.79:0/519021625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c15aa10 0x7f3a6c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 -- 172.21.15.79:0/519021625 >> 172.21.15.79:0/519021625 conn(0x7f3a6c06a670 msgr2=0x7f3a6c099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:06.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 -- 172.21.15.79:0/519021625 shutdown_connections 2023-06-08T03:51:06.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.431+0000 7f3a71134700 1 -- 172.21.15.79:0/519021625 wait complete. 2023-06-08T03:51:06.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.432+0000 7f3a71134700 1 Processor -- start 2023-06-08T03:51:06.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.432+0000 7f3a71134700 1 -- start start 2023-06-08T03:51:06.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a71134700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c0ce930 0x7f3a6c1ef500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a71134700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a71134700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c15aa10 0x7f3a6c0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a71134700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0c9910 con 0x7f3a6c15a640 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a71134700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c1efa40 con 0x7f3a6c15aa10 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a71134700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c0b61c0 con 0x7f3a6c0ce930 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a6a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a6a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35850/0 (socket says 172.21.15.79:35850) 2023-06-08T03:51:06.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a6a59c700 1 -- 172.21.15.79:0/2387618361 learned_addr learned my addr 172.21.15.79:0/2387618361 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a6ad9d700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c0ce930 0x7f3a6c1ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.433+0000 7f3a6b59e700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c15aa10 0x7f3a6c0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a6a59c700 1 -- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c0ce930 msgr2=0x7f3a6c1ef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a6a59c700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c0ce930 0x7f3a6c1ef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a6a59c700 1 -- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c15aa10 msgr2=0x7f3a6c0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a6a59c700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c15aa10 0x7f3a6c0c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a6a59c700 1 -- 172.21.15.79:0/2387618361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c009550 con 0x7f3a6c15a640 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a6a59c700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0c64c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000d800 tx=0x7f3a6000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600125f0 con 0x7f3a6c15a640 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a60012750 con 0x7f3a6c15a640 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c1f2c30 con 0x7f3a6c15a640 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60012b00 con 0x7f3a6c15a640 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.434+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c0c9250 con 0x7f3a6c15a640 2023-06-08T03:51:06.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.435+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0c90b0 con 0x7f3a6c15a640 2023-06-08T03:51:06.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.435+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a60012c60 con 0x7f3a6c15a640 2023-06-08T03:51:06.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.436+0000 7f3a537fe700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3a5806ace0 0x7f3a5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.436+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3a600896f0 con 0x7f3a6c15a640 2023-06-08T03:51:06.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.436+0000 7f3a6ad9d700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3a5806ace0 0x7f3a5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.436+0000 7f3a6ad9d700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3a5806ace0 0x7f3a5806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009c40 tx=0x7f3a5c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.440+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3a6c0c90b0 con 0x7f3a6c15a640 2023-06-08T03:51:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.627+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a6c0c90b0 con 0x7f3a6c15a640 2023-06-08T03:51:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.628+0000 7f3a537fe700 1 -- 172.21.15.79:0/2387618361 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f3a6c0c90b0 con 0x7f3a6c15a640 2023-06-08T03:51:06.631 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:06.632 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:06.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3a5806ace0 msgr2=0x7f3a5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3a5806ace0 0x7f3a5806d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009c40 tx=0x7f3a5c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:06.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 msgr2=0x7f3a6c0c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0c64c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000d800 tx=0x7f3a6000db10 comp rx=0 tx=0).stop 2023-06-08T03:51:06.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 shutdown_connections 2023-06-08T03:51:06.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a6c15aa10 0x7f3a6c0c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3a5806ace0 0x7f3a5806d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a6c0ce930 0x7f3a6c1ef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 --2- 172.21.15.79:0/2387618361 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3a6c15a640 0x7f3a6c0c64c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 >> 172.21.15.79:0/2387618361 conn(0x7f3a6c06a670 msgr2=0x7f3a6c0cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:06.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 shutdown_connections 2023-06-08T03:51:06.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.630+0000 7f3a71134700 1 -- 172.21.15.79:0/2387618361 wait complete. 2023-06-08T03:51:06.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:06.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:06.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.765+0000 7fd16bfa8700 1 Processor -- start 2023-06-08T03:51:06.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.766+0000 7fd16bfa8700 1 -- start start 2023-06-08T03:51:06.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.766+0000 7fd16bfa8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd164158ec0 0x7fd16415d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.766+0000 7fd16bfa8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd164071070 0x7fd1640cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.766+0000 7fd16bfa8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16415d8d0 con 0x7fd1640cc480 2023-06-08T03:51:06.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.766+0000 7fd16bfa8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1640cc920 con 0x7fd164071070 2023-06-08T03:51:06.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.766+0000 7fd16bfa8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16406df90 con 0x7fd164158ec0 2023-06-08T03:51:06.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd164071070 0x7fd1640cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd164071070 0x7fd1640cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35856/0 (socket says 172.21.15.79:35856) 2023-06-08T03:51:06.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 -- 172.21.15.79:0/2935719071 learned_addr learned my addr 172.21.15.79:0/2935719071 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:06.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169d44700 1 --2- 172.21.15.79:0/2935719071 >> v2:172.21.15.156:3300/0 conn(0x7fd164158ec0 0x7fd16415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 -- 172.21.15.79:0/2935719071 >> v2:172.21.15.156:3300/0 conn(0x7fd164158ec0 msgr2=0x7fd16415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 --2- 172.21.15.79:0/2935719071 >> v2:172.21.15.156:3300/0 conn(0x7fd164158ec0 0x7fd16415d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 -- 172.21.15.79:0/2935719071 >> v1:172.21.15.156:6789/0 conn(0x7fd1640cc480 legacy=0x7fd16406eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:06.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 -- 172.21.15.79:0/2935719071 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1640ca320 con 0x7fd164071070 2023-06-08T03:51:06.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd169543700 1 --2- 172.21.15.79:0/2935719071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd1640cbf40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1640c2c10 tx=0x7fd160009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7bf3479556a21 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd168d42700 1 -- 172.21.15.79:0/2935719071 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160004230 con 0x7fd164071070 2023-06-08T03:51:06.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd168d42700 1 -- 172.21.15.79:0/2935719071 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160004390 con 0x7fd164071070 2023-06-08T03:51:06.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.767+0000 7fd168d42700 1 -- 172.21.15.79:0/2935719071 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600036a0 con 0x7fd164071070 2023-06-08T03:51:06.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 -- 172.21.15.79:0/2935719071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 msgr2=0x7fd1640cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/2935719071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd1640cbf40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1640c2c10 tx=0x7fd160009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 -- 172.21.15.79:0/2935719071 shutdown_connections 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/2935719071 >> v2:172.21.15.156:3300/0 conn(0x7fd164158ec0 0x7fd16415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/2935719071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd1640cbf40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 -- 172.21.15.79:0/2935719071 >> 172.21.15.79:0/2935719071 conn(0x7fd1640b1d30 msgr2=0x7fd16406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 -- 172.21.15.79:0/2935719071 shutdown_connections 2023-06-08T03:51:06.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.768+0000 7fd16bfa8700 1 -- 172.21.15.79:0/2935719071 wait complete. 2023-06-08T03:51:06.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.769+0000 7fd16bfa8700 1 Processor -- start 2023-06-08T03:51:06.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.769+0000 7fd16bfa8700 1 -- start start 2023-06-08T03:51:06.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd164148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1640cc480 0x7fd164166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd164158ec0 0x7fd1641659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1640c9150 con 0x7fd164071070 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1640c8b90 con 0x7fd164158ec0 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1640c8e70 con 0x7fd1640cc480 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd164148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd164148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35862/0 (socket says 172.21.15.79:35862) 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 -- 172.21.15.79:0/1014076257 learned_addr learned my addr 172.21.15.79:0/1014076257 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16a545700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd164158ec0 0x7fd1641659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169543700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1640cc480 0x7fd164166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 -- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1640cc480 msgr2=0x7fd164166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1640cc480 0x7fd164166520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 -- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd164158ec0 msgr2=0x7fd1641659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd164158ec0 0x7fd1641659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 -- 172.21.15.79:0/1014076257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160009550 con 0x7fd164071070 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd169d44700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd164148a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd15400e810 tx=0x7fd15400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15400c070 con 0x7fd164071070 2023-06-08T03:51:06.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd154013610 con 0x7fd164071070 2023-06-08T03:51:06.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1640ab960 con 0x7fd164071070 2023-06-08T03:51:06.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1540139c0 con 0x7fd164071070 2023-06-08T03:51:06.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.770+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd164148f50 con 0x7fd164071070 2023-06-08T03:51:06.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.771+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1640abfb0 con 0x7fd164071070 2023-06-08T03:51:06.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.772+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd154013b20 con 0x7fd164071070 2023-06-08T03:51:06.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.772+0000 7fd15a7fc700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd14806ace0 0x7fd14806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:06.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.772+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd1540894a0 con 0x7fd164071070 2023-06-08T03:51:06.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.772+0000 7fd169543700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd14806ace0 0x7fd14806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:06.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.773+0000 7fd169543700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd14806ace0 0x7fd14806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd160009130 tx=0x7fd160008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:06.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.776+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd154057d00 con 0x7fd164071070 2023-06-08T03:51:06.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.963+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd16406fa80 con 0x7fd164071070 2023-06-08T03:51:06.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.964+0000 7fd15a7fc700 1 -- 172.21.15.79:0/1014076257 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd154057890 con 0x7fd164071070 2023-06-08T03:51:06.967 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:06.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd14806ace0 msgr2=0x7fd14806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd14806ace0 0x7fd14806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd160009130 tx=0x7fd160008030 comp rx=0 tx=0).stop 2023-06-08T03:51:06.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 msgr2=0x7fd164148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:06.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd164148a10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd15400e810 tx=0x7fd15400eb20 comp rx=0 tx=0).stop 2023-06-08T03:51:06.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 shutdown_connections 2023-06-08T03:51:06.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd164158ec0 0x7fd1641659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd164071070 0x7fd164148a10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd14806ace0 0x7fd14806d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 --2- 172.21.15.79:0/1014076257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1640cc480 0x7fd164166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:06.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 >> 172.21.15.79:0/1014076257 conn(0x7fd1640b1d30 msgr2=0x7fd16406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:06.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 shutdown_connections 2023-06-08T03:51:06.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:06.966+0000 7fd16bfa8700 1 -- 172.21.15.79:0/1014076257 wait complete. 2023-06-08T03:51:06.980 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:07.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.100+0000 7f0ffdba9700 1 Processor -- start 2023-06-08T03:51:07.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.100+0000 7f0ffdba9700 1 -- start start 2023-06-08T03:51:07.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ffdba9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0ff815c1f0 0x7f0ff806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ffdba9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0ff8069d50 0x7f0ff806a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ffdba9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff80cd140 con 0x7f0ff8071fe0 2023-06-08T03:51:07.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ffdba9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff806a710 con 0x7f0ff8069d50 2023-06-08T03:51:07.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ffdba9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff8071110 con 0x7f0ff815c1f0 2023-06-08T03:51:07.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0ff8069d50 0x7f0ff806a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff6ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0ff8071fe0 0x7f0ff80cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59346/0 (socket says 172.21.15.79:59346) 2023-06-08T03:51:07.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff6ffd700 1 -- 172.21.15.79:0/1787516490 learned_addr learned my addr 172.21.15.79:0/1787516490 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:07.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff7fff700 1 -- 172.21.15.79:0/1787516490 >> v2:172.21.15.156:3300/0 conn(0x7f0ff815c1f0 msgr2=0x7f0ff806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:07.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff77fe700 1 --2- 172.21.15.79:0/1787516490 >> v2:172.21.15.156:3300/0 conn(0x7f0ff815c1f0 0x7f0ff806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff7fff700 1 --2- 172.21.15.79:0/1787516490 >> v2:172.21.15.156:3300/0 conn(0x7f0ff815c1f0 0x7f0ff806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff67fc700 1 -- 172.21.15.79:0/1787516490 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388413490 0 0) 0x7f0ff80cd140 con 0x7f0ff8071fe0 2023-06-08T03:51:07.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff7fff700 1 -- 172.21.15.79:0/1787516490 >> v1:172.21.15.79:6789/0 conn(0x7f0ff8071fe0 legacy=0x7f0ff80cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff7fff700 1 -- 172.21.15.79:0/1787516490 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8071be0 con 0x7f0ff8069d50 2023-06-08T03:51:07.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff77fe700 1 --2- 172.21.15.79:0/1787516490 >> v2:172.21.15.156:3300/0 conn(0x7f0ff815c1f0 0x7f0ff806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:07.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff7fff700 1 --2- 172.21.15.79:0/1787516490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff8069d50 0x7f0ff806a1d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8070d60 tx=0x7f0fec009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda38e7443303f03 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.101+0000 7f0ff67fc700 1 -- 172.21.15.79:0/1787516490 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec011830 con 0x7f0ff8069d50 2023-06-08T03:51:07.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ff67fc700 1 -- 172.21.15.79:0/1787516490 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec011990 con 0x7f0ff8069d50 2023-06-08T03:51:07.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ff67fc700 1 -- 172.21.15.79:0/1787516490 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec011d20 con 0x7f0ff8069d50 2023-06-08T03:51:07.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 -- 172.21.15.79:0/1787516490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff8069d50 msgr2=0x7f0ff806a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/1787516490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff8069d50 0x7f0ff806a1d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8070d60 tx=0x7f0fec009b20 comp rx=0 tx=0).stop 2023-06-08T03:51:07.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 -- 172.21.15.79:0/1787516490 shutdown_connections 2023-06-08T03:51:07.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/1787516490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff8069d50 0x7f0ff806a1d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/1787516490 >> v2:172.21.15.156:3300/0 conn(0x7f0ff815c1f0 0x7f0ff806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 -- 172.21.15.79:0/1787516490 >> 172.21.15.79:0/1787516490 conn(0x7f0ff80b1d30 msgr2=0x7f0ff806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:07.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 -- 172.21.15.79:0/1787516490 shutdown_connections 2023-06-08T03:51:07.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.102+0000 7f0ffdba9700 1 -- 172.21.15.79:0/1787516490 wait complete. 2023-06-08T03:51:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.103+0000 7f0ffdba9700 1 Processor -- start 2023-06-08T03:51:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.103+0000 7f0ffdba9700 1 -- start start 2023-06-08T03:51:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ffdba9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8069d50 0x7f0ff80c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ffdba9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ff8071fe0 0x7f0ff80c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ffdba9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 0x7f0ff80c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ffdba9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80c7690 con 0x7f0ff815c1f0 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ffdba9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80c84f0 con 0x7f0ff8069d50 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ffdba9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80c78e0 con 0x7f0ff8071fe0 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 0x7f0ff80c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 0x7f0ff80c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35878/0 (socket says 172.21.15.79:35878) 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 -- 172.21.15.79:0/3864035347 learned_addr learned my addr 172.21.15.79:0/3864035347 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff77fe700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8069d50 0x7f0ff80c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff6ffd700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ff8071fe0 0x7f0ff80c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 -- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ff8071fe0 msgr2=0x7f0ff80c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ff8071fe0 0x7f0ff80c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 -- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8069d50 msgr2=0x7f0ff80c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8069d50 0x7f0ff80c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff7fff700 1 -- 172.21.15.79:0/3864035347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0094a0 con 0x7f0ff815c1f0 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.104+0000 7f0ff6ffd700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ff8071fe0 0x7f0ff80c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.105+0000 7f0ff7fff700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 0x7f0ff80c41b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0fec014040 tx=0x7f0fec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.105+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0045f0 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.105+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec01e070 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.105+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff80c7bc0 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.105+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003b20 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.105+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff80c8180 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.106+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff80c8660 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.106+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fec003c80 con 0x7f0ff815c1f0 2023-06-08T03:51:07.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.106+0000 7f0fdffff700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0fe406ace0 0x7f0fe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.106+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0fec08a800 con 0x7f0ff815c1f0 2023-06-08T03:51:07.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.107+0000 7f0ff77fe700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0fe406ace0 0x7f0fe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.107+0000 7f0ff77fe700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0fe406ace0 0x7f0fe406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0007510 tx=0x7f0ff00074a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.111+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fec055650 con 0x7f0ff815c1f0 2023-06-08T03:51:07.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.296+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ff8073e90 con 0x7f0ff815c1f0 2023-06-08T03:51:07.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.297+0000 7f0fdffff700 1 -- 172.21.15.79:0/3864035347 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fec058c70 con 0x7f0ff815c1f0 2023-06-08T03:51:07.301 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:07.302 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0fe406ace0 msgr2=0x7f0fe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0fe406ace0 0x7f0fe406d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0007510 tx=0x7f0ff00074a0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 msgr2=0x7f0ff80c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 0x7f0ff80c41b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0fec014040 tx=0x7f0fec004750 comp rx=0 tx=0).stop 2023-06-08T03:51:07.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 shutdown_connections 2023-06-08T03:51:07.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff815c1f0 0x7f0ff80c41b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8069d50 0x7f0ff80c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0fe406ace0 0x7f0fe406d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 --2- 172.21.15.79:0/3864035347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ff8071fe0 0x7f0ff80c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 >> 172.21.15.79:0/3864035347 conn(0x7f0ff80b1d30 msgr2=0x7f0ff815e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:07.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 shutdown_connections 2023-06-08T03:51:07.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.299+0000 7f0ffdba9700 1 -- 172.21.15.79:0/3864035347 wait complete. 2023-06-08T03:51:07.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:07.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.433+0000 7f94520dd700 1 Processor -- start 2023-06-08T03:51:07.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.433+0000 7f94520dd700 1 -- start start 2023-06-08T03:51:07.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f94520dd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f944c06d8a0 0x7f944c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f94520dd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f944c069d50 0x7f944c06a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f94520dd700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944c0cd140 con 0x7f944c15c1f0 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f94520dd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c06a730 con 0x7f944c069d50 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f94520dd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c071110 con 0x7f944c06d8a0 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f944c069d50 0x7f944c06a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f944c069d50 0x7f944c06a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35886/0 (socket says 172.21.15.79:35886) 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 -- 172.21.15.79:0/714661951 learned_addr learned my addr 172.21.15.79:0/714661951 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:07.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944affd700 1 --2- 172.21.15.79:0/714661951 >> v2:172.21.15.156:3300/0 conn(0x7f944c06d8a0 0x7f944c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 -- 172.21.15.79:0/714661951 >> v2:172.21.15.156:3300/0 conn(0x7f944c06d8a0 msgr2=0x7f944c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 --2- 172.21.15.79:0/714661951 >> v2:172.21.15.156:3300/0 conn(0x7f944c06d8a0 0x7f944c071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 -- 172.21.15.79:0/714661951 >> v1:172.21.15.156:6789/0 conn(0x7f944c15c1f0 legacy=0x7f944c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 -- 172.21.15.79:0/714661951 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c071be0 con 0x7f944c069d50 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.434+0000 7f944bfff700 1 --2- 172.21.15.79:0/714661951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c06a1f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f944c0d0670 tx=0x7f944400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f536c11fe6ab00ea server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f944a7fc700 1 -- 172.21.15.79:0/714661951 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400e070 con 0x7f944c069d50 2023-06-08T03:51:07.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f944a7fc700 1 -- 172.21.15.79:0/714661951 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944400b9c0 con 0x7f944c069d50 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f944a7fc700 1 -- 172.21.15.79:0/714661951 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400bd70 con 0x7f944c069d50 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 -- 172.21.15.79:0/714661951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 msgr2=0x7f944c06a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 --2- 172.21.15.79:0/714661951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c06a1f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f944c0d0670 tx=0x7f944400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 -- 172.21.15.79:0/714661951 shutdown_connections 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 --2- 172.21.15.79:0/714661951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c06a1f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 --2- 172.21.15.79:0/714661951 >> v2:172.21.15.156:3300/0 conn(0x7f944c06d8a0 0x7f944c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 -- 172.21.15.79:0/714661951 >> 172.21.15.79:0/714661951 conn(0x7f944c0b1d30 msgr2=0x7f944c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:07.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 -- 172.21.15.79:0/714661951 shutdown_connections 2023-06-08T03:51:07.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.435+0000 7f94520dd700 1 -- 172.21.15.79:0/714661951 wait complete. 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.436+0000 7f94520dd700 1 Processor -- start 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 -- start start 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c06d8a0 0x7f944c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f944c15c1f0 0x7f944c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0c6990 con 0x7f944c069d50 2023-06-08T03:51:07.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0c6c70 con 0x7f944c06d8a0 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f94520dd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0c6f50 con 0x7f944c15c1f0 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.437+0000 7f944b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35888/0 (socket says 172.21.15.79:35888) 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 -- 172.21.15.79:0/3551991339 learned_addr learned my addr 172.21.15.79:0/3551991339 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944bfff700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f944c15c1f0 0x7f944c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944affd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c06d8a0 0x7f944c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 -- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f944c15c1f0 msgr2=0x7f944c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f944c15c1f0 0x7f944c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 -- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c06d8a0 msgr2=0x7f944c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c06d8a0 0x7f944c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 -- 172.21.15.79:0/3551991339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444009b30 con 0x7f944c069d50 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944bfff700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f944c15c1f0 0x7f944c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944affd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c06d8a0 0x7f944c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:07.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f944b7fe700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c153820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f944000c450 tx=0x7f944000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400052d0 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440005430 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400038c0 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c0c7230 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.438+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c153d60 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.439+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c0723f0 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.440+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9440003a20 con 0x7f944c069d50 2023-06-08T03:51:07.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.440+0000 7f9433fff700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f943806ad10 0x7f943806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.440+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9440088a80 con 0x7f944c069d50 2023-06-08T03:51:07.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.440+0000 7f944affd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f943806ad10 0x7f943806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.441+0000 7f944affd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f943806ad10 0x7f943806d1c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f944c15f910 tx=0x7f943c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.444+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94400538d0 con 0x7f944c069d50 2023-06-08T03:51:07.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.630+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f944c073e90 con 0x7f944c069d50 2023-06-08T03:51:07.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.631+0000 7f9433fff700 1 -- 172.21.15.79:0/3551991339 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9440056ef0 con 0x7f944c069d50 2023-06-08T03:51:07.634 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:07.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f943806ad10 msgr2=0x7f943806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f943806ad10 0x7f943806d1c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f944c15f910 tx=0x7f943c008040 comp rx=0 tx=0).stop 2023-06-08T03:51:07.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 msgr2=0x7f944c153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c153820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f944000c450 tx=0x7f944000c810 comp rx=0 tx=0).stop 2023-06-08T03:51:07.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 shutdown_connections 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f943806ad10 0x7f943806d1c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f944c15c1f0 0x7f944c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f944c069d50 0x7f944c153820 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 --2- 172.21.15.79:0/3551991339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f944c06d8a0 0x7f944c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 >> 172.21.15.79:0/3551991339 conn(0x7f944c0b1d30 msgr2=0x7f944c0cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 shutdown_connections 2023-06-08T03:51:07.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.633+0000 7f94520dd700 1 -- 172.21.15.79:0/3551991339 wait complete. 2023-06-08T03:51:07.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:07.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.767+0000 7f02479a8700 1 Processor -- start 2023-06-08T03:51:07.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.767+0000 7f02479a8700 1 -- start start 2023-06-08T03:51:07.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f02479a8700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0240158d50 0x7f02400ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f02479a8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f02400cc690 0x7f0240071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f02479a8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02400cd7b0 con 0x7f024015d3d0 2023-06-08T03:51:07.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f02479a8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02400cda50 con 0x7f02400cc690 2023-06-08T03:51:07.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f02479a8700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02400cdd30 con 0x7f0240158d50 2023-06-08T03:51:07.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f0245f45700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f02400cc690 0x7f0240071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f0245f45700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f02400cc690 0x7f0240071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35896/0 (socket says 172.21.15.79:35896) 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f0245f45700 1 -- 172.21.15.79:0/3457745940 learned_addr learned my addr 172.21.15.79:0/3457745940 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.768+0000 7f0245744700 1 --2- 172.21.15.79:0/3457745940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0240158d50 0x7f02400ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0245f45700 1 -- 172.21.15.79:0/3457745940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0240158d50 msgr2=0x7f02400ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0245f45700 1 --2- 172.21.15.79:0/3457745940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0240158d50 0x7f02400ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0245f45700 1 -- 172.21.15.79:0/3457745940 >> v1:172.21.15.156:6789/0 conn(0x7f024015d3d0 legacy=0x7f02400cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0245f45700 1 -- 172.21.15.79:0/3457745940 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02400ce010 con 0x7f02400cc690 2023-06-08T03:51:07.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0245f45700 1 --2- 172.21.15.79:0/3457745940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02400cc690 0x7f0240071220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f023800d150 tx=0x7f023800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a86b37658c64316 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0237fff700 1 -- 172.21.15.79:0/3457745940 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238010070 con 0x7f02400cc690 2023-06-08T03:51:07.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0237fff700 1 -- 172.21.15.79:0/3457745940 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023800ddc0 con 0x7f02400cc690 2023-06-08T03:51:07.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f0237fff700 1 -- 172.21.15.79:0/3457745940 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800b310 con 0x7f02400cc690 2023-06-08T03:51:07.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f02479a8700 1 -- 172.21.15.79:0/3457745940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02400cc690 msgr2=0x7f0240071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f02479a8700 1 --2- 172.21.15.79:0/3457745940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02400cc690 0x7f0240071220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f023800d150 tx=0x7f023800d460 comp rx=0 tx=0).stop 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f02479a8700 1 -- 172.21.15.79:0/3457745940 shutdown_connections 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f02479a8700 1 --2- 172.21.15.79:0/3457745940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f02400cc690 0x7f0240071220 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f02479a8700 1 --2- 172.21.15.79:0/3457745940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0240158d50 0x7f02400ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.769+0000 7f02479a8700 1 -- 172.21.15.79:0/3457745940 >> 172.21.15.79:0/3457745940 conn(0x7f02400b1d30 msgr2=0x7f024006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.770+0000 7f02479a8700 1 -- 172.21.15.79:0/3457745940 shutdown_connections 2023-06-08T03:51:07.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.770+0000 7f02479a8700 1 -- 172.21.15.79:0/3457745940 wait complete. 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 Processor -- start 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 -- start start 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02400cc690 0x7f0240146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 0x7f0240154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f024015d3d0 0x7f0240154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02400b23a0 con 0x7f0240158d50 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02400b2bb0 con 0x7f02400cc690 2023-06-08T03:51:07.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.771+0000 7f02479a8700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02400b25f0 con 0x7f024015d3d0 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 0x7f0240154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 0x7f0240154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35906/0 (socket says 172.21.15.79:35906) 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 -- 172.21.15.79:0/37420879 learned_addr learned my addr 172.21.15.79:0/37420879 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0245f45700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f024015d3d0 0x7f0240154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0245744700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02400cc690 0x7f0240146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 -- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f024015d3d0 msgr2=0x7f0240154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f024015d3d0 0x7f0240154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 -- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02400cc690 msgr2=0x7f0240146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02400cc690 0x7f0240146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 -- 172.21.15.79:0/37420879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023800bd40 con 0x7f0240158d50 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0245f45700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f024015d3d0 0x7f0240154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0244f43700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 0x7f0240154950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f023c00c480 tx=0x7f023c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c0052d0 con 0x7f0240158d50 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c005430 con 0x7f0240158d50 2023-06-08T03:51:07.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02400b28d0 con 0x7f0240158d50 2023-06-08T03:51:07.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c0038c0 con 0x7f0240158d50 2023-06-08T03:51:07.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.772+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0240155b30 con 0x7f0240158d50 2023-06-08T03:51:07.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.773+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02400c7050 con 0x7f0240158d50 2023-06-08T03:51:07.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.773+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f023c003a20 con 0x7f0240158d50 2023-06-08T03:51:07.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.774+0000 7f0235ffb700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f022406ace0 0x7f022406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:07.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.774+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f023c088b80 con 0x7f0240158d50 2023-06-08T03:51:07.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.774+0000 7f0245744700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f022406ace0 0x7f022406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:07.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.774+0000 7f0245744700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f022406ace0 0x7f022406d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0230009930 tx=0x7f0230008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:07.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.778+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f02400c7050 con 0x7f0240158d50 2023-06-08T03:51:07.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.965+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02400c7430 con 0x7f0240158d50 2023-06-08T03:51:07.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.965+0000 7f0235ffb700 1 -- 172.21.15.79:0/37420879 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f023c017020 con 0x7f0240158d50 2023-06-08T03:51:07.969 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:07.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f022406ace0 msgr2=0x7f022406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f022406ace0 0x7f022406d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0230009930 tx=0x7f0230008040 comp rx=0 tx=0).stop 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 msgr2=0x7f0240154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 0x7f0240154950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f023c00c480 tx=0x7f023c00c840 comp rx=0 tx=0).stop 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 shutdown_connections 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f022406ace0 0x7f022406d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f024015d3d0 0x7f0240154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02400cc690 0x7f0240146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 --2- 172.21.15.79:0/37420879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0240158d50 0x7f0240154950 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:07.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 >> 172.21.15.79:0/37420879 conn(0x7f02400b1d30 msgr2=0x7f0240071120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:07.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 shutdown_connections 2023-06-08T03:51:07.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:07.967+0000 7f02479a8700 1 -- 172.21.15.79:0/37420879 wait complete. 2023-06-08T03:51:07.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:08.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.100+0000 7ff287feb700 1 Processor -- start 2023-06-08T03:51:08.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.101+0000 7ff287feb700 1 -- start start 2023-06-08T03:51:08.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.101+0000 7ff287feb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 0x7ff280070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff287feb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff280070bd0 con 0x7ff2800cd590 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff287feb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800ce560 con 0x7ff28015c0a0 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff287feb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28015c470 con 0x7ff28015fb10 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff285586700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff2800cd590 0x7ff2800cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59388/0 (socket says 172.21.15.79:59388) 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff285586700 1 -- 172.21.15.79:0/3134066152 learned_addr learned my addr 172.21.15.79:0/3134066152 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff286588700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 0x7ff280070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378730534 0 0) 0x7ff27c001110 con 0x7ff2800cd590 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff26c0035b0 con 0x7ff2800cd590 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4048679831 0 0) 0x7ff26c0035b0 con 0x7ff2800cd590 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 msgr2=0x7ff280070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 0x7ff280070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 >> v1:172.21.15.156:6789/0 conn(0x7ff28015c0a0 legacy=0x7ff2800cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280160010 con 0x7ff2800cd590 2023-06-08T03:51:08.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7ff27c001800 con 0x7ff2800cd590 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 >> v1:172.21.15.79:6789/0 conn(0x7ff2800cd590 legacy=0x7ff2800cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff26c003a30 0x7ff26c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 0x7ff26c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c004d90 0x7ff26c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28015c470 con 0x7ff26c004d90 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c00bb20 con 0x7ff26c003a30 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c00bce0 con 0x7ff28015fb10 2023-06-08T03:51:08.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff286588700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c004d90 0x7ff26c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff285586700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 0x7ff26c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.102+0000 7ff285d87700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff26c003a30 0x7ff26c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 -- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 msgr2=0x7ff26c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff28015fb10 0x7ff26c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 -- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff26c003a30 msgr2=0x7ff26c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff26c003a30 0x7ff26c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 -- 172.21.15.79:0/3134066152 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0035b0 con 0x7ff26c004d90 2023-06-08T03:51:08.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c004d90 0x7ff26c005200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff278009fc0 tx=0x7ff26c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165ec04973c98a58 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800ec30 con 0x7ff26c004d90 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800f040 con 0x7ff26c004d90 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff284d85700 1 -- 172.21.15.79:0/3134066152 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278014480 con 0x7ff26c004d90 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff287feb700 1 -- 172.21.15.79:0/3134066152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c004d90 msgr2=0x7ff26c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff287feb700 1 --2- 172.21.15.79:0/3134066152 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff26c004d90 0x7ff26c005200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff278009fc0 tx=0x7ff26c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff286588700 1 -- 172.21.15.79:0/3134066152 reap_dead start 2023-06-08T03:51:08.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff287feb700 1 -- 172.21.15.79:0/3134066152 shutdown_connections 2023-06-08T03:51:08.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff287feb700 1 -- 172.21.15.79:0/3134066152 >> 172.21.15.79:0/3134066152 conn(0x7ff28006e1e0 msgr2=0x7ff28006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:08.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff287feb700 1 -- 172.21.15.79:0/3134066152 shutdown_connections 2023-06-08T03:51:08.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.103+0000 7ff287feb700 1 -- 172.21.15.79:0/3134066152 wait complete. 2023-06-08T03:51:08.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 Processor -- start 2023-06-08T03:51:08.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 -- start start 2023-06-08T03:51:08.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 0x7ff2800c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800c46e0 0x7ff2800b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2800b5160 0x7ff2800b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800c4b50 con 0x7ff2800b4850 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800b09d0 con 0x7ff2800c46e0 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff287feb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800b5b10 con 0x7ff2800b5160 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff285d87700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 0x7ff2800c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff285d87700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 0x7ff2800c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35922/0 (socket says 172.21.15.79:35922) 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.105+0000 7ff285d87700 1 -- 172.21.15.79:0/634993765 learned_addr learned my addr 172.21.15.79:0/634993765 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff286588700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2800b5160 0x7ff2800b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285586700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800c46e0 0x7ff2800b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285d87700 1 -- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2800b5160 msgr2=0x7ff2800b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285d87700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2800b5160 0x7ff2800b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285d87700 1 -- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800c46e0 msgr2=0x7ff2800b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285d87700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800c46e0 0x7ff2800b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285d87700 1 -- 172.21.15.79:0/634993765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800b5da0 con 0x7ff2800b4850 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff285d87700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 0x7ff2800c41a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff27000f820 tx=0x7ff27000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700052e0 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270005440 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2800b0c20 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700038c0 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.106+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2800b11e0 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.107+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2800b13a0 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.108+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff270003a20 con 0x7ff2800b4850 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.108+0000 7ff2767fc700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff26406ace0 0x7ff26406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.108+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff270089f60 con 0x7ff2800b4850 2023-06-08T03:51:08.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.108+0000 7ff285586700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff26406ace0 0x7ff26406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.109+0000 7ff285586700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff26406ace0 0x7ff26406d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff27c001050 tx=0x7ff27c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.112+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff270054e90 con 0x7ff2800b4850 2023-06-08T03:51:08.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.296+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff280073e90 con 0x7ff2800b4850 2023-06-08T03:51:08.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.297+0000 7ff2767fc700 1 -- 172.21.15.79:0/634993765 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff2700584b0 con 0x7ff2800b4850 2023-06-08T03:51:08.301 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff26406ace0 msgr2=0x7ff26406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff26406ace0 0x7ff26406d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff27c001050 tx=0x7ff27c009040 comp rx=0 tx=0).stop 2023-06-08T03:51:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 msgr2=0x7ff2800c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 0x7ff2800c41a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff27000f820 tx=0x7ff27000fbe0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 shutdown_connections 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff26406ace0 0x7ff26406d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2800b5160 0x7ff2800b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff2800b4850 0x7ff2800c41a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 --2- 172.21.15.79:0/634993765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800c46e0 0x7ff2800b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 >> 172.21.15.79:0/634993765 conn(0x7ff28006e1e0 msgr2=0x7ff28015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 shutdown_connections 2023-06-08T03:51:08.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.299+0000 7ff287feb700 1 -- 172.21.15.79:0/634993765 wait complete. 2023-06-08T03:51:08.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:08.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.429+0000 7f9f995dd700 1 Processor -- start 2023-06-08T03:51:08.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 -- start start 2023-06-08T03:51:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9f940c9630 0x7f9f94158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f94158f80 0x7f9f940ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9f940cd9b0 0x7f9f94159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f94159a10 con 0x7f9f940c9630 2023-06-08T03:51:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f940cc810 con 0x7f9f940cd9b0 2023-06-08T03:51:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f995dd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f94159450 con 0x7f9f94158f80 2023-06-08T03:51:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f92ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9f940c9630 0x7f9f94158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f92ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9f940c9630 0x7f9f94158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35930/0 (socket says 172.21.15.79:35930) 2023-06-08T03:51:08.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.430+0000 7f9f92ffd700 1 -- 172.21.15.79:0/2128747746 learned_addr learned my addr 172.21.15.79:0/2128747746 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f92ffd700 1 -- 172.21.15.79:0/2128747746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f94158f80 msgr2=0x7f9f940ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f937fe700 1 --2- 172.21.15.79:0/2128747746 >> v2:172.21.15.156:3300/0 conn(0x7f9f940cd9b0 0x7f9f94159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f927fc700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f94158f80 0x7f9f940ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f94158f80 0x7f9f940ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f92ffd700 1 -- 172.21.15.79:0/2128747746 >> v2:172.21.15.156:3300/0 conn(0x7f9f940cd9b0 msgr2=0x7f9f94159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:08.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/2128747746 >> v2:172.21.15.156:3300/0 conn(0x7f9f940cd9b0 0x7f9f94159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f92ffd700 1 -- 172.21.15.79:0/2128747746 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94159730 con 0x7f9f940c9630 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f937fe700 1 --2- 172.21.15.79:0/2128747746 >> v2:172.21.15.156:3300/0 conn(0x7f9f940cd9b0 0x7f9f94159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f927fc700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f94158f80 0x7f9f940ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f940c9630 0x7f9f94158a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9f9406fe40 tx=0x7f9f88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482663a100158025 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f91ffb700 1 -- 172.21.15.79:0/2128747746 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800b870 con 0x7f9f940c9630 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f91ffb700 1 -- 172.21.15.79:0/2128747746 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8800b9d0 con 0x7f9f940c9630 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.431+0000 7f9f91ffb700 1 -- 172.21.15.79:0/2128747746 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800bd60 con 0x7f9f940c9630 2023-06-08T03:51:08.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 -- 172.21.15.79:0/2128747746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f940c9630 msgr2=0x7f9f94158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f940c9630 0x7f9f94158a40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9f9406fe40 tx=0x7f9f88009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 -- 172.21.15.79:0/2128747746 shutdown_connections 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 --2- 172.21.15.79:0/2128747746 >> v2:172.21.15.156:3300/0 conn(0x7f9f940cd9b0 0x7f9f94159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f940c9630 0x7f9f94158a40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 --2- 172.21.15.79:0/2128747746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f94158f80 0x7f9f940ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 -- 172.21.15.79:0/2128747746 >> 172.21.15.79:0/2128747746 conn(0x7f9f94069d50 msgr2=0x7f9f9406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:08.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 -- 172.21.15.79:0/2128747746 shutdown_connections 2023-06-08T03:51:08.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.432+0000 7f9f995dd700 1 -- 172.21.15.79:0/2128747746 wait complete. 2023-06-08T03:51:08.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.433+0000 7f9f995dd700 1 Processor -- start 2023-06-08T03:51:08.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.433+0000 7f9f995dd700 1 -- start start 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f940c9630 0x7f9f940c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f940cd9b0 0x7f9f9414cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 0x7f9f94150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940c4500 con 0x7f9f94158f80 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940c2b00 con 0x7f9f940cd9b0 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940c47a0 con 0x7f9f940c9630 2023-06-08T03:51:08.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 0x7f9f94150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 0x7f9f94150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35940/0 (socket says 172.21.15.79:35940) 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 -- 172.21.15.79:0/1953794033 learned_addr learned my addr 172.21.15.79:0/1953794033 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f940c9630 0x7f9f940c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f927fc700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f940cd9b0 0x7f9f9414cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 -- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f940c9630 msgr2=0x7f9f940c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f940c9630 0x7f9f940c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 -- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f940cd9b0 msgr2=0x7f9f9414cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f940cd9b0 0x7f9f9414cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 -- 172.21.15.79:0/1953794033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88009550 con 0x7f9f94158f80 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f927fc700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f940cd9b0 0x7f9f9414cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f940c9630 0x7f9f940c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f937fe700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 0x7f9f94150a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00cdd0 tx=0x7f9f8c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00ad70 con 0x7f9f94158f80 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c015440 con 0x7f9f94158f80 2023-06-08T03:51:08.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f940c4a80 con 0x7f9f94158f80 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0157f0 con 0x7f9f94158f80 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.434+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9414d9f0 con 0x7f9f94158f80 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.435+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f940c4c40 con 0x7f9f94158f80 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.436+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f8c00d430 con 0x7f9f94158f80 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.436+0000 7f9f7b7fe700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f8006ace0 0x7f9f8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.436+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9f8c089d80 con 0x7f9f94158f80 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.436+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f8006ace0 0x7f9f8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.437+0000 7f9f92ffd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f8006ace0 0x7f9f8006d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f88000c00 tx=0x7f9f88008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.440+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f8c054ac0 con 0x7f9f94158f80 2023-06-08T03:51:08.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.627+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f940c4360 con 0x7f9f94158f80 2023-06-08T03:51:08.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.628+0000 7f9f7b7fe700 1 -- 172.21.15.79:0/1953794033 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f9f940c4360 con 0x7f9f94158f80 2023-06-08T03:51:08.632 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:08.633 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:08.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f8006ace0 msgr2=0x7f9f8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f8006ace0 0x7f9f8006d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f88000c00 tx=0x7f9f88008030 comp rx=0 tx=0).stop 2023-06-08T03:51:08.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 msgr2=0x7f9f94150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 0x7f9f94150a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00cdd0 tx=0x7f9f8c00ceb0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 shutdown_connections 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f94158f80 0x7f9f94150a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9f8006ace0 0x7f9f8006d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9f940c9630 0x7f9f940c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 --2- 172.21.15.79:0/1953794033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9f940cd9b0 0x7f9f9414cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.635 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 >> 172.21.15.79:0/1953794033 conn(0x7f9f94069d50 msgr2=0x7f9f94155a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 shutdown_connections 2023-06-08T03:51:08.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.630+0000 7f9f995dd700 1 -- 172.21.15.79:0/1953794033 wait complete. 2023-06-08T03:51:08.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:08.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:08.768 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 Processor -- start 2023-06-08T03:51:08.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 -- start start 2023-06-08T03:51:08.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11a415c1f0 0x7f11a406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f11a4069d50 0x7f11a406a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a40cd140 con 0x7f11a4071fe0 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a406a710 con 0x7f11a415c1f0 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11aa92e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a4071110 con 0x7f11a4069d50 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11a3fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11a415c1f0 0x7f11a406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11a37fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f11a4071fe0 0x7f11a40cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59420/0 (socket says 172.21.15.79:59420) 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.765+0000 7f11a37fe700 1 -- 172.21.15.79:0/2819124480 learned_addr learned my addr 172.21.15.79:0/2819124480 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a8ecb700 1 --2- 172.21.15.79:0/2819124480 >> v2:172.21.15.156:3300/0 conn(0x7f11a4069d50 0x7f11a406a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a3fff700 1 -- 172.21.15.79:0/2819124480 >> v2:172.21.15.156:3300/0 conn(0x7f11a4069d50 msgr2=0x7f11a406a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a3fff700 1 --2- 172.21.15.79:0/2819124480 >> v2:172.21.15.156:3300/0 conn(0x7f11a4069d50 0x7f11a406a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a3fff700 1 -- 172.21.15.79:0/2819124480 >> v1:172.21.15.79:6789/0 conn(0x7f11a4071fe0 legacy=0x7f11a40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a3fff700 1 -- 172.21.15.79:0/2819124480 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4071be0 con 0x7f11a415c1f0 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a2ffd700 1 -- 172.21.15.79:0/2819124480 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178639161 0 0) 0x7f11a40cd140 con 0x7f11a4071fe0 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a3fff700 1 --2- 172.21.15.79:0/2819124480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a406d8a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f11a4070e40 tx=0x7f1194009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3102ffe7b3d224fd server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a2ffd700 1 -- 172.21.15.79:0/2819124480 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194011830 con 0x7f11a415c1f0 2023-06-08T03:51:08.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a2ffd700 1 -- 172.21.15.79:0/2819124480 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194011990 con 0x7f11a415c1f0 2023-06-08T03:51:08.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.766+0000 7f11a2ffd700 1 -- 172.21.15.79:0/2819124480 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194011d20 con 0x7f11a415c1f0 2023-06-08T03:51:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 -- 172.21.15.79:0/2819124480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 msgr2=0x7f11a406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 --2- 172.21.15.79:0/2819124480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a406d8a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f11a4070e40 tx=0x7f1194009b20 comp rx=0 tx=0).stop 2023-06-08T03:51:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 -- 172.21.15.79:0/2819124480 shutdown_connections 2023-06-08T03:51:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 --2- 172.21.15.79:0/2819124480 >> v2:172.21.15.156:3300/0 conn(0x7f11a4069d50 0x7f11a406a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 --2- 172.21.15.79:0/2819124480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a406d8a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 -- 172.21.15.79:0/2819124480 >> 172.21.15.79:0/2819124480 conn(0x7f11a40b1d30 msgr2=0x7f11a406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 -- 172.21.15.79:0/2819124480 shutdown_connections 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.767+0000 7f11aa92e700 1 -- 172.21.15.79:0/2819124480 wait complete. 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.768+0000 7f11aa92e700 1 Processor -- start 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.768+0000 7f11aa92e700 1 -- start start 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11aa92e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a4069d50 0x7f11a41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11aa92e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a4071fe0 0x7f11a40c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11aa92e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11aa92e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4158730 con 0x7f11a415c1f0 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11aa92e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4157ab0 con 0x7f11a4071fe0 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11aa92e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4157d90 con 0x7f11a4069d50 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a414f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35954/0 (socket says 172.21.15.79:35954) 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 -- 172.21.15.79:0/3964482110 learned_addr learned my addr 172.21.15.79:0/3964482110 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a3fff700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a4069d50 0x7f11a41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a37fe700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a4071fe0 0x7f11a40c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 -- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a4069d50 msgr2=0x7f11a41574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a4069d50 0x7f11a41574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 -- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a4071fe0 msgr2=0x7f11a40c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a4071fe0 0x7f11a40c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 -- 172.21.15.79:0/3964482110 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940094a0 con 0x7f11a415c1f0 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a8ecb700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a414f6e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f119c007e60 tx=0x7f119c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.769+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c00ed30 con 0x7f11a415c1f0 2023-06-08T03:51:08.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.770+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c014420 con 0x7f11a415c1f0 2023-06-08T03:51:08.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.770+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a4158070 con 0x7f11a415c1f0 2023-06-08T03:51:08.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.770+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0147d0 con 0x7f11a415c1f0 2023-06-08T03:51:08.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.770+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a40c8810 con 0x7f11a415c1f0 2023-06-08T03:51:08.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.770+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a4073e90 con 0x7f11a415c1f0 2023-06-08T03:51:08.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.771+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f119c014930 con 0x7f11a415c1f0 2023-06-08T03:51:08.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.771+0000 7f11a0ff9700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f118406ace0 0x7f118406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:08.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.771+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f119c089d10 con 0x7f11a415c1f0 2023-06-08T03:51:08.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.772+0000 7f11a3fff700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f118406ace0 0x7f118406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:08.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.772+0000 7f11a3fff700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f118406ace0 0x7f118406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11940090b0 tx=0x7f119401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:08.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.776+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f119c054c10 con 0x7f11a415c1f0 2023-06-08T03:51:08.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.965+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f11a406ed70 con 0x7f11a415c1f0 2023-06-08T03:51:08.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.965+0000 7f11a0ff9700 1 -- 172.21.15.79:0/3964482110 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f119c058230 con 0x7f11a415c1f0 2023-06-08T03:51:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:08.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f118406ace0 msgr2=0x7f118406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f118406ace0 0x7f118406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11940090b0 tx=0x7f119401c040 comp rx=0 tx=0).stop 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 msgr2=0x7f11a414f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a414f6e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f119c007e60 tx=0x7f119c00e3c0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 shutdown_connections 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11a415c1f0 0x7f11a414f6e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f118406ace0 0x7f118406d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11a4069d50 0x7f11a41574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 --2- 172.21.15.79:0/3964482110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11a4071fe0 0x7f11a40c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:08.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 >> 172.21.15.79:0/3964482110 conn(0x7f11a40b1d30 msgr2=0x7f11a40b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:08.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 shutdown_connections 2023-06-08T03:51:08.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:08.967+0000 7f11aa92e700 1 -- 172.21.15.79:0/3964482110 wait complete. 2023-06-08T03:51:08.983 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:09.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.101+0000 7f86c09aa700 1 Processor -- start 2023-06-08T03:51:09.105 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.101+0000 7f86c09aa700 1 -- start start 2023-06-08T03:51:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86c09aa700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f86bc06a460 0x7f86bc0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86c09aa700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc06dfa0 con 0x7f86bc06daa0 2023-06-08T03:51:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86c09aa700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc06ee30 con 0x7f86bc0b5000 2023-06-08T03:51:09.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86c09aa700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc15a360 con 0x7f86bc06a460 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b9d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f86bc06daa0 0x7f86bc069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59436/0 (socket says 172.21.15.79:59436) 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b9d9b700 1 -- 172.21.15.79:0/3973674454 learned_addr learned my addr 172.21.15.79:0/3973674454 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86bad9d700 1 --2- 172.21.15.79:0/3973674454 >> v2:172.21.15.156:3300/0 conn(0x7f86bc06a460 0x7f86bc0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785767134 0 0) 0x7f86b0001110 con 0x7f86bc06daa0 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86a40035b0 con 0x7f86bc06daa0 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 661349407 0 0) 0x7f86a40035b0 con 0x7f86bc06daa0 2023-06-08T03:51:09.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 >> v2:172.21.15.156:3300/0 conn(0x7f86bc06a460 msgr2=0x7f86bc0c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 --2- 172.21.15.79:0/3973674454 >> v2:172.21.15.156:3300/0 conn(0x7f86bc06a460 0x7f86bc0c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 >> v1:172.21.15.156:6789/0 conn(0x7f86bc0b5000 legacy=0x7f86bc06f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc071f30 con 0x7f86bc06daa0 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f86b0001800 con 0x7f86bc06daa0 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 >> v1:172.21.15.79:6789/0 conn(0x7f86bc06daa0 legacy=0x7f86bc069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4003a00 0x7f86a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86a4004850 0x7f86a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.102+0000 7f86b959a700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86a4005200 0x7f86a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc15a360 con 0x7f86a4005200 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a400c080 con 0x7f86a4004850 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a400c240 con 0x7f86a4003a00 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86a4005200 0x7f86a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b9d9b700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86a4004850 0x7f86a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86ba59c700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4003a00 0x7f86a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 -- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4003a00 msgr2=0x7f86a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4003a00 0x7f86a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 -- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86a4004850 msgr2=0x7f86a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86a4004850 0x7f86a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 -- 172.21.15.79:0/3973674454 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a40035b0 con 0x7f86a4005200 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 -- 172.21.15.79:0/3973674454 reap_dead start 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86bad9d700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86a4005200 0x7f86a400ba10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009fc0 tx=0x7f86a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3e9ce2a062e5c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4003a20 con 0x7f86a4005200 2023-06-08T03:51:09.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b400f040 con 0x7f86a4005200 2023-06-08T03:51:09.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.103+0000 7f86b959a700 1 -- 172.21.15.79:0/3973674454 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4003de0 con 0x7f86a4005200 2023-06-08T03:51:09.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 -- 172.21.15.79:0/3973674454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86a4005200 msgr2=0x7f86a400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86a4005200 0x7f86a400ba10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009fc0 tx=0x7f86a4005880 comp rx=0 tx=0).stop 2023-06-08T03:51:09.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 -- 172.21.15.79:0/3973674454 shutdown_connections 2023-06-08T03:51:09.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 --2- 172.21.15.79:0/3973674454 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86a4005200 0x7f86a400ba10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 -- 172.21.15.79:0/3973674454 >> 172.21.15.79:0/3973674454 conn(0x7f86bc0b1d30 msgr2=0x7f86bc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:09.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 -- 172.21.15.79:0/3973674454 shutdown_connections 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.104+0000 7f86c09aa700 1 -- 172.21.15.79:0/3973674454 wait complete. 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.105+0000 7f86c09aa700 1 Processor -- start 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.105+0000 7f86c09aa700 1 -- start start 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86c09aa700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4005200 0x7f86bc158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86c09aa700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 0x7f86bc15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86c09aa700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc15af50 0x7f86bc1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86c09aa700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc15d2e0 con 0x7f86bc1f0160 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86c09aa700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc15b3c0 con 0x7f86bc15af50 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86c09aa700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc15c3c0 con 0x7f86a4005200 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 0x7f86bc15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 0x7f86bc15aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35972/0 (socket says 172.21.15.79:35972) 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 -- 172.21.15.79:0/1672380616 learned_addr learned my addr 172.21.15.79:0/1672380616 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86ba59c700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4005200 0x7f86bc158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86bad9d700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc15af50 0x7f86bc1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 -- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4005200 msgr2=0x7f86bc158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4005200 0x7f86bc158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 -- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc15af50 msgr2=0x7f86bc1f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc15af50 0x7f86bc1f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.106+0000 7f86b9d9b700 1 -- 172.21.15.79:0/1672380616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc15c660 con 0x7f86bc1f0160 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.107+0000 7f86b9d9b700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 0x7f86bc15aa10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f86b0007310 tx=0x7f86b00076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.107+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00077f0 con 0x7f86bc1f0160 2023-06-08T03:51:09.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.107+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b000f870 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.107+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc15c940 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.107+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000fc20 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.107+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc15cf00 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.108+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc15d420 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.108+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86b000fd80 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.109+0000 7f86aaffd700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f869c06ace0 0x7f869c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.109+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f86b00895c0 con 0x7f86bc1f0160 2023-06-08T03:51:09.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.109+0000 7f86ba59c700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f869c06ace0 0x7f869c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.109+0000 7f86ba59c700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f869c06ace0 0x7f869c06d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0c5cc0 tx=0x7f86ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.113+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86bc15d420 con 0x7f86bc1f0160 2023-06-08T03:51:09.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.299+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86bc06dfa0 con 0x7f86bc1f0160 2023-06-08T03:51:09.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.299+0000 7f86aaffd700 1 -- 172.21.15.79:0/1672380616 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86bc06dfa0 con 0x7f86bc1f0160 2023-06-08T03:51:09.303 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:09.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f869c06ace0 msgr2=0x7f869c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f869c06ace0 0x7f869c06d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0c5cc0 tx=0x7f86ac009040 comp rx=0 tx=0).stop 2023-06-08T03:51:09.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 msgr2=0x7f86bc15aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 0x7f86bc15aa10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f86b0007310 tx=0x7f86b00076d0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 shutdown_connections 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86bc15af50 0x7f86bc1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f869c06ace0 0x7f869c06d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86a4005200 0x7f86bc158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 --2- 172.21.15.79:0/1672380616 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f86bc1f0160 0x7f86bc15aa10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 >> 172.21.15.79:0/1672380616 conn(0x7f86bc0b1d30 msgr2=0x7f86bc0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:09.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 shutdown_connections 2023-06-08T03:51:09.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.301+0000 7f86c09aa700 1 -- 172.21.15.79:0/1672380616 wait complete. 2023-06-08T03:51:09.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:09.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.435+0000 7f4638457700 1 Processor -- start 2023-06-08T03:51:09.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.435+0000 7f4638457700 1 -- start start 2023-06-08T03:51:09.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f4638457700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f4638457700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 0x7f46300b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f4638457700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46300b4f40 con 0x7f46300d0740 2023-06-08T03:51:09.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f4638457700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46300d2e20 con 0x7f46300b5a50 2023-06-08T03:51:09.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f4638457700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300d3100 con 0x7f46300d1220 2023-06-08T03:51:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46369f4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 0x7f46300b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46359f2700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46369f4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 0x7f46300b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58876/0 (socket says 172.21.15.79:58876) 2023-06-08T03:51:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46369f4700 1 -- 172.21.15.79:0/2590964672 learned_addr learned my addr 172.21.15.79:0/2590964672 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46359f2700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:50302/0 (socket says 172.21.15.79:50302) 2023-06-08T03:51:09.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46361f3700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f46300d0740 0x7f46300d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:33384/0 (socket says 172.21.15.79:33384) 2023-06-08T03:51:09.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46359f2700 1 -- 172.21.15.79:0/2590964672 >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 msgr2=0x7f46300b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46359f2700 1 --2- 172.21.15.79:0/2590964672 >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 0x7f46300b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46359f2700 1 -- 172.21.15.79:0/2590964672 >> v1:172.21.15.156:6789/0 conn(0x7f46300d0740 legacy=0x7f46300d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46359f2700 1 -- 172.21.15.79:0/2590964672 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46300d33e0 con 0x7f46300d1220 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.436+0000 7f46369f4700 1 --2- 172.21.15.79:0/2590964672 >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 0x7f46300b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f46359f2700 1 --2- 172.21.15.79:0/2590964672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300cd1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4620009840 tx=0x7f4620009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f7fbee8f9a0d980 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f46351f1700 1 -- 172.21.15.79:0/2590964672 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620011830 con 0x7f46300d1220 2023-06-08T03:51:09.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f46351f1700 1 -- 172.21.15.79:0/2590964672 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4620011990 con 0x7f46300d1220 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f46351f1700 1 -- 172.21.15.79:0/2590964672 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620011d20 con 0x7f46300d1220 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 -- 172.21.15.79:0/2590964672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 msgr2=0x7f46300cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 --2- 172.21.15.79:0/2590964672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300cd1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4620009840 tx=0x7f4620009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 -- 172.21.15.79:0/2590964672 shutdown_connections 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 --2- 172.21.15.79:0/2590964672 >> v2:172.21.15.156:3300/0 conn(0x7f46300b5a50 0x7f46300b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 --2- 172.21.15.79:0/2590964672 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300cd1c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 -- 172.21.15.79:0/2590964672 >> 172.21.15.79:0/2590964672 conn(0x7f4630071480 msgr2=0x7f4630071850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:09.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.437+0000 7f4638457700 1 -- 172.21.15.79:0/2590964672 shutdown_connections 2023-06-08T03:51:09.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.438+0000 7f4638457700 1 -- 172.21.15.79:0/2590964672 wait complete. 2023-06-08T03:51:09.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 Processor -- start 2023-06-08T03:51:09.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 -- start start 2023-06-08T03:51:09.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46300b5a50 0x7f46301532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 0x7f46300c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463015f930 con 0x7f46300d0740 2023-06-08T03:51:09.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630142da0 con 0x7f46300b5a50 2023-06-08T03:51:09.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.439+0000 7f4638457700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463015f650 con 0x7f46300d1220 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 0x7f46300c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 0x7f46300c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35986/0 (socket says 172.21.15.79:35986) 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 -- 172.21.15.79:0/3779758690 learned_addr learned my addr 172.21.15.79:0/3779758690 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46369f4700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46361f3700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46300b5a50 0x7f46301532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 -- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 msgr2=0x7f46300c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 -- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46300b5a50 msgr2=0x7f46301532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46300b5a50 0x7f46301532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 -- 172.21.15.79:0/3779758690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628009b30 con 0x7f46300d0740 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f46359f2700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 0x7f46300c8180 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f46200090b0 tx=0x7f4620004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200036a0 con 0x7f46300d0740 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462001e070 con 0x7f46300d0740 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46200094a0 con 0x7f46300d0740 2023-06-08T03:51:09.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620003a80 con 0x7f46300d0740 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.440+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46300cb0d0 con 0x7f46300d0740 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.441+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4630073e90 con 0x7f46300d0740 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.441+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4620003be0 con 0x7f46300d0740 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.442+0000 7f4626ffd700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f461406ace0 0x7f461406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.442+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f462008a5c0 con 0x7f46300d0740 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.442+0000 7f46361f3700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f461406ace0 0x7f461406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.442+0000 7f46361f3700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f461406ace0 0x7f461406d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f60 tx=0x7f462c007ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.446+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4620055410 con 0x7f46300d0740 2023-06-08T03:51:09.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.630+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4630066790 con 0x7f46300d0740 2023-06-08T03:51:09.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.630+0000 7f4626ffd700 1 -- 172.21.15.79:0/3779758690 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4620020020 con 0x7f46300d0740 2023-06-08T03:51:09.634 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:09.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f461406ace0 msgr2=0x7f461406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f461406ace0 0x7f461406d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f60 tx=0x7f462c007ed0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 msgr2=0x7f46300c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 0x7f46300c8180 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f46200090b0 tx=0x7f4620004750 comp rx=0 tx=0).stop 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 shutdown_connections 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f461406ace0 0x7f461406d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46300d1220 0x7f46300c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46300b5a50 0x7f46301532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 --2- 172.21.15.79:0/3779758690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46300d0740 0x7f46300c8180 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 >> 172.21.15.79:0/3779758690 conn(0x7f4630071480 msgr2=0x7f4630071850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:09.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 shutdown_connections 2023-06-08T03:51:09.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.632+0000 7f4638457700 1 -- 172.21.15.79:0/3779758690 wait complete. 2023-06-08T03:51:09.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:09.769 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.765+0000 7f916b21e700 1 Processor -- start 2023-06-08T03:51:09.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.766+0000 7f916b21e700 1 -- start start 2023-06-08T03:51:09.770 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.766+0000 7f916b21e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f916406a460 0x7f9164071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.766+0000 7f916b21e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91640cd140 con 0x7f916406d8a0 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.766+0000 7f916b21e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9164071110 con 0x7f916415c1f0 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.766+0000 7f916b21e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9164071be0 con 0x7f916406a460 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.766+0000 7f91697bb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f916406a460 0x7f9164071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f9163fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f916406d8a0 0x7f9164069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59470/0 (socket says 172.21.15.79:59470) 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f9163fff700 1 -- 172.21.15.79:0/3409992836 learned_addr learned my addr 172.21.15.79:0/3409992836 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:09.771 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91697bb700 1 -- 172.21.15.79:0/3409992836 >> v1:172.21.15.156:6789/0 conn(0x7f916415c1f0 legacy=0x7f91640cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:09.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91697bb700 1 -- 172.21.15.79:0/3409992836 >> v1:172.21.15.79:6789/0 conn(0x7f916406d8a0 legacy=0x7f9164069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.772 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91697bb700 1 -- 172.21.15.79:0/3409992836 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640cd390 con 0x7f916406a460 2023-06-08T03:51:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91697bb700 1 --2- 172.21.15.79:0/3409992836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f9164071fe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f915c009d90 tx=0x7f915c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6de9668c363f19 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91637fe700 1 -- 172.21.15.79:0/3409992836 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00e070 con 0x7f916406a460 2023-06-08T03:51:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91637fe700 1 -- 172.21.15.79:0/3409992836 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c00bb90 con 0x7f916406a460 2023-06-08T03:51:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.767+0000 7f91637fe700 1 -- 172.21.15.79:0/3409992836 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c003a20 con 0x7f916406a460 2023-06-08T03:51:09.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 -- 172.21.15.79:0/3409992836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 msgr2=0x7f9164071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 --2- 172.21.15.79:0/3409992836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f9164071fe0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f915c009d90 tx=0x7f915c00b330 comp rx=0 tx=0).stop 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 -- 172.21.15.79:0/3409992836 shutdown_connections 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 --2- 172.21.15.79:0/3409992836 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f9164071fe0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 -- 172.21.15.79:0/3409992836 >> 172.21.15.79:0/3409992836 conn(0x7f91640b1d30 msgr2=0x7f916406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 -- 172.21.15.79:0/3409992836 shutdown_connections 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.768+0000 7f916b21e700 1 -- 172.21.15.79:0/3409992836 wait complete. 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.769+0000 7f916b21e700 1 Processor -- start 2023-06-08T03:51:09.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.769+0000 7f916b21e700 1 -- start start 2023-06-08T03:51:09.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f916b21e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f91640c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f916b21e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f916406d8a0 0x7f916414b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f916b21e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f916415c1f0 0x7f91641f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f916b21e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91640c8080 con 0x7f916406a460 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f916b21e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916415d9c0 con 0x7f916406d8a0 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f916b21e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91640c99b0 con 0x7f916415c1f0 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f91640c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f91640c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35998/0 (socket says 172.21.15.79:35998) 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 -- 172.21.15.79:0/3650591795 learned_addr learned my addr 172.21.15.79:0/3650591795 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:09.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9163fff700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f916406d8a0 0x7f916414b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 -- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f916415c1f0 msgr2=0x7f91641f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f916415c1f0 0x7f91641f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 -- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f916406d8a0 msgr2=0x7f916414b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f916406d8a0 0x7f916414b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 -- 172.21.15.79:0/3650591795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c009a70 con 0x7f916406a460 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9163fff700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f916406d8a0 0x7f916414b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:09.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f9168fba700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f91640c9470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f915800b4a0 tx=0x7f915800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158011830 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.770+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9158011990 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.771+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916415dc30 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.771+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158011d40 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.771+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916415e470 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.771+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916415e690 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.772+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f915801a400 con 0x7f916406a460 2023-06-08T03:51:09.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.772+0000 7f91617fa700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f914806ace0 0x7f914806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.772+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f91580894f0 con 0x7f916406a460 2023-06-08T03:51:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.773+0000 7f9163fff700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f914806ace0 0x7f914806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.773+0000 7f9163fff700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f914806ace0 0x7f914806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f916414c0c0 tx=0x7f9150009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:09.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.777+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91580542b0 con 0x7f916406a460 2023-06-08T03:51:09.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.963+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9164073e90 con 0x7f916406a460 2023-06-08T03:51:09.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.963+0000 7f91617fa700 1 -- 172.21.15.79:0/3650591795 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91580578d0 con 0x7f916406a460 2023-06-08T03:51:09.968 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:09.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f914806ace0 msgr2=0x7f914806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f914806ace0 0x7f914806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f916414c0c0 tx=0x7f9150009040 comp rx=0 tx=0).stop 2023-06-08T03:51:09.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 msgr2=0x7f91640c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f91640c9470 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f915800b4a0 tx=0x7f915800b860 comp rx=0 tx=0).stop 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 shutdown_connections 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f914806ace0 0x7f914806d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f916415c1f0 0x7f91641f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916406a460 0x7f91640c9470 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 --2- 172.21.15.79:0/3650591795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f916406d8a0 0x7f916414b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:09.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 >> 172.21.15.79:0/3650591795 conn(0x7f91640b1d30 msgr2=0x7f91640ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:09.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 shutdown_connections 2023-06-08T03:51:09.971 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:09.965+0000 7f916b21e700 1 -- 172.21.15.79:0/3650591795 wait complete. 2023-06-08T03:51:09.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.099+0000 7f54c7478700 1 Processor -- start 2023-06-08T03:51:10.103 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.099+0000 7f54c7478700 1 -- start start 2023-06-08T03:51:10.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c7478700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 0x7f54c0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c7478700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c00cd140 con 0x7f54c0069d50 2023-06-08T03:51:10.106 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c7478700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c0071110 con 0x7f54c015c1f0 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c7478700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c0071be0 con 0x7f54c006d8a0 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c5a15700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f54c0069d50 0x7f54c00cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59486/0 (socket says 172.21.15.79:59486) 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c5a15700 1 -- 172.21.15.79:0/3193155342 learned_addr learned my addr 172.21.15.79:0/3193155342 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54c4a13700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 0x7f54c0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932167549 0 0) 0x7f54c00cd140 con 0x7f54c0069d50 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.100+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ac0035b0 con 0x7f54c0069d50 2023-06-08T03:51:10.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009975059 0 0) 0x7f54c0071110 con 0x7f54c015c1f0 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c00cd140 con 0x7f54c015c1f0 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3369873083 0 0) 0x7f54ac0035b0 con 0x7f54c0069d50 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 msgr2=0x7f54c0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 0x7f54c0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 >> v1:172.21.15.156:6789/0 conn(0x7f54c015c1f0 legacy=0x7f54c00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00cd390 con 0x7f54c0069d50 2023-06-08T03:51:10.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f54b80019f0 con 0x7f54c0069d50 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 >> v1:172.21.15.79:6789/0 conn(0x7f54c0069d50 legacy=0x7f54c00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 0x7f54ac003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ac004170 0x7f54ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c0071be0 con 0x7f54ac004d90 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac00bc50 con 0x7f54ac004170 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac00be10 con 0x7f54c006d8a0 2023-06-08T03:51:10.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5214700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 0x7f54ac003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c4a13700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ac004170 0x7f54ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 -- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 msgr2=0x7f54ac003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c006d8a0 0x7f54ac003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 -- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ac004170 msgr2=0x7f54ac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ac004170 0x7f54ac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 -- 172.21.15.79:0/3193155342 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac0035b0 con 0x7f54ac004d90 2023-06-08T03:51:10.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c4a13700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54ac004170 0x7f54ac004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54c5a15700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54ac005200 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54b8002230 tx=0x7f54ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566445a3c8a06c9f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80072d0 con 0x7f54ac004d90 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8007430 con 0x7f54ac004d90 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.101+0000 7f54b7fff700 1 -- 172.21.15.79:0/3193155342 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80077e0 con 0x7f54ac004d90 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c7478700 1 -- 172.21.15.79:0/3193155342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 msgr2=0x7f54ac005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c7478700 1 --2- 172.21.15.79:0/3193155342 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54ac005200 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54b8002230 tx=0x7f54ac005880 comp rx=0 tx=0).stop 2023-06-08T03:51:10.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c5a15700 1 -- 172.21.15.79:0/3193155342 reap_dead start 2023-06-08T03:51:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c7478700 1 -- 172.21.15.79:0/3193155342 shutdown_connections 2023-06-08T03:51:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c7478700 1 -- 172.21.15.79:0/3193155342 >> 172.21.15.79:0/3193155342 conn(0x7f54c00b1d30 msgr2=0x7f54c006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c7478700 1 -- 172.21.15.79:0/3193155342 shutdown_connections 2023-06-08T03:51:10.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.102+0000 7f54c7478700 1 -- 172.21.15.79:0/3193155342 wait complete. 2023-06-08T03:51:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.103+0000 7f54c7478700 1 Processor -- start 2023-06-08T03:51:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.103+0000 7f54c7478700 1 -- start start 2023-06-08T03:51:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c7478700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54c0158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c7478700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c014b9d0 0x7f54c00c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c7478700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c01f2490 0x7f54c015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c7478700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00c64a0 con 0x7f54ac004d90 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c7478700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00c7ec0 con 0x7f54c014b9d0 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c7478700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00c8160 con 0x7f54c01f2490 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54c0158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54c0158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36018/0 (socket says 172.21.15.79:36018) 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 -- 172.21.15.79:0/1024499132 learned_addr learned my addr 172.21.15.79:0/1024499132 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:10.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 -- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c01f2490 msgr2=0x7f54c015de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c01f2490 0x7f54c015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 -- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c014b9d0 msgr2=0x7f54c00c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c014b9d0 0x7f54c00c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.104+0000 7f54c5214700 1 -- 172.21.15.79:0/1024499132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c00c8440 con 0x7f54ac004d90 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.105+0000 7f54c5214700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54c0158b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f54b000f7f0 tx=0x7f54b000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.105+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b00052e0 con 0x7f54ac004d90 2023-06-08T03:51:10.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.105+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b0005440 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.105+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c00c8720 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.105+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b00038c0 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.105+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c00c60b0 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.106+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c00c6270 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.106+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54b0003a20 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.107+0000 7f54b5ffb700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f54a406ace0 0x7f54a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.107+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f54b0089660 con 0x7f54ac004d90 2023-06-08T03:51:10.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.107+0000 7f54c4a13700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f54a406ace0 0x7f54a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.107+0000 7f54c4a13700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f54a406ace0 0x7f54a406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0099b0 tx=0x7f54bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.111+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f54b0054420 con 0x7f54ac004d90 2023-06-08T03:51:10.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.299+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f54c00c8910 con 0x7f54ac004d90 2023-06-08T03:51:10.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.299+0000 7f54b5ffb700 1 -- 172.21.15.79:0/1024499132 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f54b0057a40 con 0x7f54ac004d90 2023-06-08T03:51:10.304 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:10.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f54a406ace0 msgr2=0x7f54a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f54a406ace0 0x7f54a406d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0099b0 tx=0x7f54bc008040 comp rx=0 tx=0).stop 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 msgr2=0x7f54c0158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54c0158b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f54b000f7f0 tx=0x7f54b000fbb0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 shutdown_connections 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f54a406ace0 0x7f54a406d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c01f2490 0x7f54c015de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54ac004d90 0x7f54c0158b40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 --2- 172.21.15.79:0/1024499132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c014b9d0 0x7f54c00c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 >> 172.21.15.79:0/1024499132 conn(0x7f54c00b1d30 msgr2=0x7f54c00729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:10.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 shutdown_connections 2023-06-08T03:51:10.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.301+0000 7f54c7478700 1 -- 172.21.15.79:0/1024499132 wait complete. 2023-06-08T03:51:10.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:10.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.433+0000 7fd8d5060700 1 Processor -- start 2023-06-08T03:51:10.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.433+0000 7fd8d5060700 1 -- start start 2023-06-08T03:51:10.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8d5060700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8d0159200 0x7fd8d00cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8d5060700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8d5060700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d00cacb0 con 0x7fd8d0158e30 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8d5060700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d00caef0 con 0x7fd8d0159200 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8d5060700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00cb1d0 con 0x7fd8d015d730 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ce59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8d0159200 0x7fd8d00cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ced9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd8d0158e30 0x7fd8d00ca5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59502/0 (socket says 172.21.15.79:59502) 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ced9d700 1 -- 172.21.15.79:0/2981983202 learned_addr learned my addr 172.21.15.79:0/2981983202 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:10.439 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8cf59e700 1 --2- 172.21.15.79:0/2981983202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ce59c700 1 -- 172.21.15.79:0/2981983202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 msgr2=0x7fd8d00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ce59c700 1 --2- 172.21.15.79:0/2981983202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d00cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ce59c700 1 -- 172.21.15.79:0/2981983202 >> v1:172.21.15.79:6789/0 conn(0x7fd8d0158e30 legacy=0x7fd8d00ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ce59c700 1 -- 172.21.15.79:0/2981983202 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00cb4b0 con 0x7fd8d0159200 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.434+0000 7fd8ce59c700 1 --2- 172.21.15.79:0/2981983202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d00cbd80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009840 tx=0x7fd8c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1dfc1018a9ca5d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8cdd9b700 1 -- 172.21.15.79:0/2981983202 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0011830 con 0x7fd8d0159200 2023-06-08T03:51:10.440 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8cdd9b700 1 -- 172.21.15.79:0/2981983202 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0011990 con 0x7fd8d0159200 2023-06-08T03:51:10.441 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8cdd9b700 1 -- 172.21.15.79:0/2981983202 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0011d20 con 0x7fd8d0159200 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 -- 172.21.15.79:0/2981983202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 msgr2=0x7fd8d00cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 --2- 172.21.15.79:0/2981983202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d00cbd80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009840 tx=0x7fd8c0009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 -- 172.21.15.79:0/2981983202 shutdown_connections 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 --2- 172.21.15.79:0/2981983202 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 --2- 172.21.15.79:0/2981983202 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d00cbd80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 -- 172.21.15.79:0/2981983202 >> 172.21.15.79:0/2981983202 conn(0x7fd8d00b1d20 msgr2=0x7fd8d006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:10.443 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 -- 172.21.15.79:0/2981983202 shutdown_connections 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.435+0000 7fd8d5060700 1 -- 172.21.15.79:0/2981983202 wait complete. 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 Processor -- start 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 -- start start 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8d0158e30 0x7fd8d01495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c7750 con 0x7fd8d0159200 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c6b40 con 0x7fd8d0158e30 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.437+0000 7fd8d5060700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d00c6e20 con 0x7fd8d015d730 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d006eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36034/0 (socket says 172.21.15.79:36034) 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 -- 172.21.15.79:0/3164317352 learned_addr learned my addr 172.21.15.79:0/3164317352 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8cf59e700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ced9d700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8d0158e30 0x7fd8d01495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 -- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 msgr2=0x7fd8d006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d006f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 -- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8d0158e30 msgr2=0x7fd8d01495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8d0158e30 0x7fd8d01495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 -- 172.21.15.79:0/3164317352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00094a0 con 0x7fd8d0159200 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8ce59c700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d006eaf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00090b0 tx=0x7fd8c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00038a0 con 0x7fd8d0159200 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c001f070 con 0x7fd8d0159200 2023-06-08T03:51:10.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d00c7100 con 0x7fd8d0159200 2023-06-08T03:51:10.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0003c60 con 0x7fd8d0159200 2023-06-08T03:51:10.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.438+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d00b23a0 con 0x7fd8d0159200 2023-06-08T03:51:10.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.439+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d0073e90 con 0x7fd8d0159200 2023-06-08T03:51:10.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.440+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8c001a430 con 0x7fd8d0159200 2023-06-08T03:51:10.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.440+0000 7fd8b77fe700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd8bc06ace0 0x7fd8bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.440+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd8c0021070 con 0x7fd8d0159200 2023-06-08T03:51:10.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.440+0000 7fd8ced9d700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd8bc06ace0 0x7fd8bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.441+0000 7fd8ced9d700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd8bc06ace0 0x7fd8bc06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4005910 tx=0x7fd8c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.444+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8c0056080 con 0x7fd8d0159200 2023-06-08T03:51:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.633+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8d00b54f0 con 0x7fd8d0159200 2023-06-08T03:51:10.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.634+0000 7fd8b77fe700 1 -- 172.21.15.79:0/3164317352 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fd8c00596a0 con 0x7fd8d0159200 2023-06-08T03:51:10.639 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:10.639 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:10.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd8bc06ace0 msgr2=0x7fd8bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd8bc06ace0 0x7fd8bc06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4005910 tx=0x7fd8c4009040 comp rx=0 tx=0).stop 2023-06-08T03:51:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 msgr2=0x7fd8d006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d006eaf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00090b0 tx=0x7fd8c0004750 comp rx=0 tx=0).stop 2023-06-08T03:51:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 shutdown_connections 2023-06-08T03:51:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd8bc06ace0 0x7fd8bc06d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8d015d730 0x7fd8d006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8d0158e30 0x7fd8d01495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 --2- 172.21.15.79:0/3164317352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8d0159200 0x7fd8d006eaf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 >> 172.21.15.79:0/3164317352 conn(0x7fd8d00b1d20 msgr2=0x7fd8d0071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 shutdown_connections 2023-06-08T03:51:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.637+0000 7fd8d5060700 1 -- 172.21.15.79:0/3164317352 wait complete. 2023-06-08T03:51:10.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:10.653 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:10.774 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.770+0000 7faa90c1c700 1 Processor -- start 2023-06-08T03:51:10.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.770+0000 7faa90c1c700 1 -- start start 2023-06-08T03:51:10.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa90c1c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faa8c0cc900 0x7faa8c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa90c1c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0ae150 0x7faa8c0cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa90c1c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c0cd210 con 0x7faa8c0cb1e0 2023-06-08T03:51:10.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa90c1c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c069d50 con 0x7faa8c0cc900 2023-06-08T03:51:10.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa90c1c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c15a460 con 0x7faa8c0ae150 2023-06-08T03:51:10.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faa8c0cc900 0x7faa8c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8ad9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7faa8c0cb1e0 0x7faa8c06aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59520/0 (socket says 172.21.15.79:59520) 2023-06-08T03:51:10.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8ad9d700 1 -- 172.21.15.79:0/1001573807 learned_addr learned my addr 172.21.15.79:0/1001573807 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:10.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8a59c700 1 -- 172.21.15.79:0/1001573807 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0ae150 msgr2=0x7faa8c0cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:10.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa89d9b700 1 --2- 172.21.15.79:0/1001573807 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0ae150 0x7faa8c0cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8a59c700 1 --2- 172.21.15.79:0/1001573807 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0ae150 0x7faa8c0cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8a59c700 1 -- 172.21.15.79:0/1001573807 >> v1:172.21.15.79:6789/0 conn(0x7faa8c0cb1e0 legacy=0x7faa8c06aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8959a700 1 -- 172.21.15.79:0/1001573807 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611218371 0 0) 0x7faa8c0cd210 con 0x7faa8c0cb1e0 2023-06-08T03:51:10.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8a59c700 1 -- 172.21.15.79:0/1001573807 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c069f60 con 0x7faa8c0cc900 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8a59c700 1 --2- 172.21.15.79:0/1001573807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0cc900 0x7faa8c06c770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009840 tx=0x7faa7c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e4617649e6936b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8959a700 1 -- 172.21.15.79:0/1001573807 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c011830 con 0x7faa8c0cc900 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8959a700 1 -- 172.21.15.79:0/1001573807 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c011990 con 0x7faa8c0cc900 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.772+0000 7faa8959a700 1 -- 172.21.15.79:0/1001573807 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c011d20 con 0x7faa8c0cc900 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 -- 172.21.15.79:0/1001573807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0cc900 msgr2=0x7faa8c06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 --2- 172.21.15.79:0/1001573807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0cc900 0x7faa8c06c770 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009840 tx=0x7faa7c009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 -- 172.21.15.79:0/1001573807 shutdown_connections 2023-06-08T03:51:10.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 --2- 172.21.15.79:0/1001573807 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0cc900 0x7faa8c06c770 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 --2- 172.21.15.79:0/1001573807 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0ae150 0x7faa8c0cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 -- 172.21.15.79:0/1001573807 >> 172.21.15.79:0/1001573807 conn(0x7faa8c0ced30 msgr2=0x7faa8c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:10.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 -- 172.21.15.79:0/1001573807 shutdown_connections 2023-06-08T03:51:10.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.773+0000 7faa90c1c700 1 -- 172.21.15.79:0/1001573807 wait complete. 2023-06-08T03:51:10.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.774+0000 7faa90c1c700 1 Processor -- start 2023-06-08T03:51:10.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.774+0000 7faa90c1c700 1 -- start start 2023-06-08T03:51:10.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa90c1c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 0x7faa8c158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa90c1c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0cb1e0 0x7faa8c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa90c1c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0cc900 0x7faa8c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa90c1c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1f3120 con 0x7faa8c0ae150 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa90c1c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1e2390 con 0x7faa8c0cb1e0 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa90c1c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c1f3370 con 0x7faa8c0cc900 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 0x7faa8c158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 0x7faa8c158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36046/0 (socket says 172.21.15.79:36046) 2023-06-08T03:51:10.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 -- 172.21.15.79:0/3975215087 learned_addr learned my addr 172.21.15.79:0/3975215087 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa89d9b700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0cb1e0 0x7faa8c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8ad9d700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0cc900 0x7faa8c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 -- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0cc900 msgr2=0x7faa8c0b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0cc900 0x7faa8c0b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 -- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0cb1e0 msgr2=0x7faa8c0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0cb1e0 0x7faa8c0b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8a59c700 1 -- 172.21.15.79:0/3975215087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0094a0 con 0x7faa8c0ae150 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa89d9b700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0cb1e0 0x7faa8c0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.775+0000 7faa8ad9d700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0cc900 0x7faa8c0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa8a59c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 0x7faa8c158a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000c00 tx=0x7faa7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c0045f0 con 0x7faa8c0ae150 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c01f070 con 0x7faa8c0ae150 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c1f3650 con 0x7faa8c0ae150 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c003ae0 con 0x7faa8c0ae150 2023-06-08T03:51:10.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.776+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c158f90 con 0x7faa8c0ae150 2023-06-08T03:51:10.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.777+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c159120 con 0x7faa8c0ae150 2023-06-08T03:51:10.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.777+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa7c003c40 con 0x7faa8c0ae150 2023-06-08T03:51:10.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.778+0000 7faa7affd700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faa6c06ace0 0x7faa6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:10.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.778+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faa7c021070 con 0x7faa8c0ae150 2023-06-08T03:51:10.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.778+0000 7faa89d9b700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faa6c06ace0 0x7faa6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:10.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.778+0000 7faa89d9b700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faa6c06ace0 0x7faa6c06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0b1400 tx=0x7faa80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:10.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.782+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faa7c056060 con 0x7faa8c0ae150 2023-06-08T03:51:10.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.968+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faa8c06f1d0 con 0x7faa8c0ae150 2023-06-08T03:51:10.973 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.969+0000 7faa7affd700 1 -- 172.21.15.79:0/3975215087 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7faa7c059680 con 0x7faa8c0ae150 2023-06-08T03:51:10.973 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:10.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faa6c06ace0 msgr2=0x7faa6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faa6c06ace0 0x7faa6c06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0b1400 tx=0x7faa80008040 comp rx=0 tx=0).stop 2023-06-08T03:51:10.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 msgr2=0x7faa8c158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:10.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 0x7faa8c158a50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000c00 tx=0x7faa7c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 shutdown_connections 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7faa6c06ace0 0x7faa6c06d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa8c0cc900 0x7faa8c0b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa8c0ae150 0x7faa8c158a50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 --2- 172.21.15.79:0/3975215087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa8c0cb1e0 0x7faa8c0b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 >> 172.21.15.79:0/3975215087 conn(0x7faa8c0ced30 msgr2=0x7faa8c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 shutdown_connections 2023-06-08T03:51:10.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:10.971+0000 7faa90c1c700 1 -- 172.21.15.79:0/3975215087 wait complete. 2023-06-08T03:51:10.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:11.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.104+0000 7f181e43b700 1 Processor -- start 2023-06-08T03:51:11.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.105+0000 7f181e43b700 1 -- start start 2023-06-08T03:51:11.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.105+0000 7f181e43b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f181815c1f0 0x7f181806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.105+0000 7f181e43b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f18180cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.105+0000 7f181e43b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180cd140 con 0x7f18180cf130 2023-06-08T03:51:11.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.105+0000 7f181e43b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1818071110 con 0x7f181815c1f0 2023-06-08T03:51:11.110 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.105+0000 7f181e43b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1818071be0 con 0x7f1818071fe0 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f181815c1f0 0x7f181806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f18177fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f18180cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f181815c1f0 0x7f181806d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:58948/0 (socket says 172.21.15.79:58948) 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f18177fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f18180cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:50378/0 (socket says 172.21.15.79:50378) 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f181c9d8700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f18180cf130 0x7f1818069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:33464/0 (socket says 172.21.15.79:33464) 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 -- 172.21.15.79:0/436473393 learned_addr learned my addr 172.21.15.79:0/436473393 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 -- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 msgr2=0x7f18180cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:11.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 --2- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f18180cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 -- 172.21.15.79:0/436473393 >> v1:172.21.15.156:6789/0 conn(0x7f18180cf130 legacy=0x7f1818069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 -- 172.21.15.79:0/436473393 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180cd390 con 0x7f181815c1f0 2023-06-08T03:51:11.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1816ffd700 1 -- 172.21.15.79:0/436473393 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179669443 0 0) 0x7f18180cd140 con 0x7f18180cf130 2023-06-08T03:51:11.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1817fff700 1 --2- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f181806d8a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f180c0098f0 tx=0x7f180c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b3f01d81e4ea3b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1816ffd700 1 -- 172.21.15.79:0/436473393 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c011830 con 0x7f181815c1f0 2023-06-08T03:51:11.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1816ffd700 1 -- 172.21.15.79:0/436473393 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c011990 con 0x7f181815c1f0 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.106+0000 7f1816ffd700 1 -- 172.21.15.79:0/436473393 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c011d20 con 0x7f181815c1f0 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 -- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 msgr2=0x7f181806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 --2- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f181806d8a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f180c0098f0 tx=0x7f180c009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 -- 172.21.15.79:0/436473393 shutdown_connections 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 --2- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f181806d8a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 --2- 172.21.15.79:0/436473393 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f18180cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 -- 172.21.15.79:0/436473393 >> 172.21.15.79:0/436473393 conn(0x7f18180b1d30 msgr2=0x7f181806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:11.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 -- 172.21.15.79:0/436473393 shutdown_connections 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.107+0000 7f181e43b700 1 -- 172.21.15.79:0/436473393 wait complete. 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.108+0000 7f181e43b700 1 Processor -- start 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.108+0000 7f181e43b700 1 -- start start 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181e43b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f1818149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181e43b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 0x7f18180c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181e43b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f18180c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181e43b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181814a080 con 0x7f18180cf130 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181e43b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180c8170 con 0x7f181815c1f0 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181e43b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b23a0 con 0x7f1818071fe0 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f18177fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 0x7f18180c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f18177fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 0x7f18180c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36062/0 (socket says 172.21.15.79:36062) 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f18177fe700 1 -- 172.21.15.79:0/2606384650 learned_addr learned my addr 172.21.15.79:0/2606384650 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f181c9d8700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f18180c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.109+0000 7f1817fff700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f1818149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f18177fe700 1 -- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 msgr2=0x7f1818149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f18177fe700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f1818149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f18177fe700 1 -- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 msgr2=0x7f18180c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f18177fe700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f18180c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f18177fe700 1 -- 172.21.15.79:0/2606384650 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009550 con 0x7f18180cf130 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f18177fe700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 0x7f18180c85d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f180800e870 tx=0x7f180800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808009590 con 0x7f18180cf130 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1808013680 con 0x7f18180cf130 2023-06-08T03:51:11.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18180b2680 con 0x7f18180cf130 2023-06-08T03:51:11.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18080139f0 con 0x7f18180cf130 2023-06-08T03:51:11.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.110+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18180b2c40 con 0x7f18180cf130 2023-06-08T03:51:11.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.111+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1818073e90 con 0x7f18180cf130 2023-06-08T03:51:11.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.112+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1808013b50 con 0x7f18180cf130 2023-06-08T03:51:11.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.112+0000 7f1814ff9700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f180406ace0 0x7f180406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.112+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f18080895e0 con 0x7f18180cf130 2023-06-08T03:51:11.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.112+0000 7f1817fff700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f180406ace0 0x7f180406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.113+0000 7f1817fff700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f180406ace0 0x7f180406d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f180c014040 tx=0x7f180c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.117+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1808057e30 con 0x7f18180cf130 2023-06-08T03:51:11.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.301+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f181806ed70 con 0x7f18180cf130 2023-06-08T03:51:11.306 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.302+0000 7f1814ff9700 1 -- 172.21.15.79:0/2606384650 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18080579c0 con 0x7f18180cf130 2023-06-08T03:51:11.306 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:11.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f180406ace0 msgr2=0x7f180406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f180406ace0 0x7f180406d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f180c014040 tx=0x7f180c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:11.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 msgr2=0x7f18180c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 0x7f18180c85d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f180800e870 tx=0x7f180800eb80 comp rx=0 tx=0).stop 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 shutdown_connections 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f181815c1f0 0x7f18180c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f180406ace0 0x7f180406d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1818071fe0 0x7f1818149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 --2- 172.21.15.79:0/2606384650 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18180cf130 0x7f18180c85d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 >> 172.21.15.79:0/2606384650 conn(0x7f18180b1d30 msgr2=0x7f181806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 shutdown_connections 2023-06-08T03:51:11.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.304+0000 7f181e43b700 1 -- 172.21.15.79:0/2606384650 wait complete. 2023-06-08T03:51:11.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 Processor -- start 2023-06-08T03:51:11.442 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 -- start start 2023-06-08T03:51:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f82e80ccef0 0x7f82e80d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82e80d09b0 0x7f82e80b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e80b5940 con 0x7f82e815f820 2023-06-08T03:51:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80cef00 con 0x7f82e80ccef0 2023-06-08T03:51:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.438+0000 7f82ed833700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80d0dd0 con 0x7f82e80d09b0 2023-06-08T03:51:11.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f82e80ccef0 0x7f82e80d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f82e80ccef0 0x7f82e80d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36070/0 (socket says 172.21.15.79:36070) 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 -- 172.21.15.79:0/3861861778 learned_addr learned my addr 172.21.15.79:0/3861861778 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e77fe700 1 --2- 172.21.15.79:0/3861861778 >> v2:172.21.15.156:3300/0 conn(0x7f82e80d09b0 0x7f82e80b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 -- 172.21.15.79:0/3861861778 >> v2:172.21.15.156:3300/0 conn(0x7f82e80d09b0 msgr2=0x7f82e80b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 --2- 172.21.15.79:0/3861861778 >> v2:172.21.15.156:3300/0 conn(0x7f82e80d09b0 0x7f82e80b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 -- 172.21.15.79:0/3861861778 >> v1:172.21.15.156:6789/0 conn(0x7f82e815f820 legacy=0x7f82e80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:11.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 -- 172.21.15.79:0/3861861778 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80ce660 con 0x7f82e80ccef0 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e77fe700 1 --2- 172.21.15.79:0/3861861778 >> v2:172.21.15.156:3300/0 conn(0x7f82e80d09b0 0x7f82e80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e67fc700 1 --2- 172.21.15.79:0/3861861778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e80d0470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f82e809c350 tx=0x7f82dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51df7b3706ddf7b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e5ffb700 1 -- 172.21.15.79:0/3861861778 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00b870 con 0x7f82e80ccef0 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e5ffb700 1 -- 172.21.15.79:0/3861861778 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00b9d0 con 0x7f82e80ccef0 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.439+0000 7f82e5ffb700 1 -- 172.21.15.79:0/3861861778 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00bd60 con 0x7f82e80ccef0 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 -- 172.21.15.79:0/3861861778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 msgr2=0x7f82e80d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 --2- 172.21.15.79:0/3861861778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e80d0470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f82e809c350 tx=0x7f82dc009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 -- 172.21.15.79:0/3861861778 shutdown_connections 2023-06-08T03:51:11.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 --2- 172.21.15.79:0/3861861778 >> v2:172.21.15.156:3300/0 conn(0x7f82e80d09b0 0x7f82e80b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 --2- 172.21.15.79:0/3861861778 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e80d0470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 -- 172.21.15.79:0/3861861778 >> 172.21.15.79:0/3861861778 conn(0x7f82e809bc40 msgr2=0x7f82e80d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:11.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 -- 172.21.15.79:0/3861861778 shutdown_connections 2023-06-08T03:51:11.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.440+0000 7f82ed833700 1 -- 172.21.15.79:0/3861861778 wait complete. 2023-06-08T03:51:11.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.441+0000 7f82ed833700 1 Processor -- start 2023-06-08T03:51:11.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.441+0000 7f82ed833700 1 -- start start 2023-06-08T03:51:11.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82ed833700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82ed833700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82e80d09b0 0x7f82e81f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82ed833700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82e815f820 0x7f82e80d2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82ed833700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80c9970 con 0x7f82e80ccef0 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82ed833700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80cca10 con 0x7f82e815f820 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82ed833700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80c9bc0 con 0x7f82e80d09b0 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36074/0 (socket says 172.21.15.79:36074) 2023-06-08T03:51:11.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 -- 172.21.15.79:0/4184777184 learned_addr learned my addr 172.21.15.79:0/4184777184 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:11.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e77fe700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82e815f820 0x7f82e80d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e67fc700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82e80d09b0 0x7f82e81f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 -- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82e80d09b0 msgr2=0x7f82e81f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82e80d09b0 0x7f82e81f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 -- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82e815f820 msgr2=0x7f82e80d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82e815f820 0x7f82e80d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 -- 172.21.15.79:0/4184777184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc009550 con 0x7f82e80ccef0 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.442+0000 7f82e6ffd700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e815ae50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f82d800c4d0 tx=0x7f82d800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.443+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80052d0 con 0x7f82e80ccef0 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.443+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8005430 con 0x7f82e80ccef0 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.443+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e80c9ea0 con 0x7f82e80ccef0 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.443+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80038f0 con 0x7f82e80ccef0 2023-06-08T03:51:11.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.443+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e81f2e90 con 0x7f82e80ccef0 2023-06-08T03:51:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.444+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e81f3050 con 0x7f82e80ccef0 2023-06-08T03:51:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.444+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82d8003a50 con 0x7f82e80ccef0 2023-06-08T03:51:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.444+0000 7f82cffff700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f82d406ace0 0x7f82d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.444+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f82d8088b60 con 0x7f82e80ccef0 2023-06-08T03:51:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.444+0000 7f82e67fc700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f82d406ace0 0x7f82d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.445+0000 7f82e67fc700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f82d406ace0 0x7f82d406d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f82dc014040 tx=0x7f82dc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.449+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f82d8053930 con 0x7f82e80ccef0 2023-06-08T03:51:11.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.632+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82e8073e90 con 0x7f82e80ccef0 2023-06-08T03:51:11.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.633+0000 7f82cffff700 1 -- 172.21.15.79:0/4184777184 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82d8056f50 con 0x7f82e80ccef0 2023-06-08T03:51:11.637 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.634+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f82d406ace0 msgr2=0x7f82d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.634+0000 7f82ed833700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f82d406ace0 0x7f82d406d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f82dc014040 tx=0x7f82dc008030 comp rx=0 tx=0).stop 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.634+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 msgr2=0x7f82e815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.634+0000 7f82ed833700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e815ae50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f82d800c4d0 tx=0x7f82d800c7e0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 shutdown_connections 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82e815f820 0x7f82e80d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f82e80ccef0 0x7f82e815ae50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f82d406ace0 0x7f82d406d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 --2- 172.21.15.79:0/4184777184 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82e80d09b0 0x7f82e81f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 >> 172.21.15.79:0/4184777184 conn(0x7f82e809bc40 msgr2=0x7f82e815e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 shutdown_connections 2023-06-08T03:51:11.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.635+0000 7f82ed833700 1 -- 172.21.15.79:0/4184777184 wait complete. 2023-06-08T03:51:11.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.769+0000 7fda81047700 1 Processor -- start 2023-06-08T03:51:11.773 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.769+0000 7fda81047700 1 -- start start 2023-06-08T03:51:11.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda81047700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fda7c0d0060 0x7fda7c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.775 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda81047700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fda7c0cd080 0x7fda7c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda81047700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c0b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda81047700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c0cbea0 con 0x7fda7c0d0060 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda81047700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c0cda30 con 0x7fda7c0cd080 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda81047700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0cae50 con 0x7fda7c0b20f0 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fda7c0d0060 0x7fda7c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fda7c0d0060 0x7fda7c0af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36082/0 (socket says 172.21.15.79:36082) 2023-06-08T03:51:11.776 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 -- 172.21.15.79:0/2126288022 learned_addr learned my addr 172.21.15.79:0/2126288022 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7b59e700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c0b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7a59c700 1 --2- 172.21.15.79:0/2126288022 >> v2:172.21.15.156:3300/0 conn(0x7fda7c0cd080 0x7fda7c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 -- 172.21.15.79:0/2126288022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 msgr2=0x7fda7c0b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c0b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 -- 172.21.15.79:0/2126288022 >> v2:172.21.15.156:3300/0 conn(0x7fda7c0cd080 msgr2=0x7fda7c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 --2- 172.21.15.79:0/2126288022 >> v2:172.21.15.156:3300/0 conn(0x7fda7c0cd080 0x7fda7c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7ad9d700 1 -- 172.21.15.79:0/2126288022 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c15d930 con 0x7fda7c0d0060 2023-06-08T03:51:11.777 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.770+0000 7fda7b59e700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c0b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda7ad9d700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0d0060 0x7fda7c0af7d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0b62c0 tx=0x7fda70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdbd3bf34e954d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda79d9b700 1 -- 172.21.15.79:0/2126288022 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000b860 con 0x7fda7c0d0060 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda79d9b700 1 -- 172.21.15.79:0/2126288022 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7000b9c0 con 0x7fda7c0d0060 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda79d9b700 1 -- 172.21.15.79:0/2126288022 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7000bd50 con 0x7fda7c0d0060 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 -- 172.21.15.79:0/2126288022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0d0060 msgr2=0x7fda7c0af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0d0060 0x7fda7c0af7d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0b62c0 tx=0x7fda70009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 -- 172.21.15.79:0/2126288022 shutdown_connections 2023-06-08T03:51:11.778 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c0b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 --2- 172.21.15.79:0/2126288022 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0d0060 0x7fda7c0af7d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 --2- 172.21.15.79:0/2126288022 >> v2:172.21.15.156:3300/0 conn(0x7fda7c0cd080 0x7fda7c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 -- 172.21.15.79:0/2126288022 >> 172.21.15.79:0/2126288022 conn(0x7fda7c06e1e0 msgr2=0x7fda7c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 -- 172.21.15.79:0/2126288022 shutdown_connections 2023-06-08T03:51:11.779 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.771+0000 7fda81047700 1 -- 172.21.15.79:0/2126288022 wait complete. 2023-06-08T03:51:11.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.773+0000 7fda81047700 1 Processor -- start 2023-06-08T03:51:11.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.773+0000 7fda81047700 1 -- start start 2023-06-08T03:51:11.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.773+0000 7fda81047700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c1e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 0x7fda7c0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c0d0060 0x7fda7c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1f4d10 con 0x7fda7c0cd080 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1f2ac0 con 0x7fda7c0d0060 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1e4f30 con 0x7fda7c0b20f0 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 0x7fda7c0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 0x7fda7c0b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36090/0 (socket says 172.21.15.79:36090) 2023-06-08T03:51:11.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 -- 172.21.15.79:0/3026057799 learned_addr learned my addr 172.21.15.79:0/3026057799 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7ad9d700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c1e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7b59e700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c0d0060 0x7fda7c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 -- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 msgr2=0x7fda7c1e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c1e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 -- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c0d0060 msgr2=0x7fda7c0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c0d0060 0x7fda7c0b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 -- 172.21.15.79:0/3026057799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda70009550 con 0x7fda7c0cd080 2023-06-08T03:51:11.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda7a59c700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 0x7fda7c0b3f90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e7e0 tx=0x7fda6c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c013610 con 0x7fda7c0cd080 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c013770 con 0x7fda7c0cd080 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c1e5210 con 0x7fda7c0cd080 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c013ae0 con 0x7fda7c0cd080 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.774+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c1e57d0 con 0x7fda7c0cd080 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.775+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c1f2ce0 con 0x7fda7c0cd080 2023-06-08T03:51:11.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.776+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda6c013c40 con 0x7fda7c0cd080 2023-06-08T03:51:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.776+0000 7fda637fe700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fda6806ace0 0x7fda6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.776+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda6c089310 con 0x7fda7c0cd080 2023-06-08T03:51:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.776+0000 7fda7ad9d700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fda6806ace0 0x7fda6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.777+0000 7fda7ad9d700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fda6806ace0 0x7fda6806d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda70008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:11.785 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.780+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda7c1f2ce0 con 0x7fda7c0cd080 2023-06-08T03:51:11.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.968+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda7c1f2c80 con 0x7fda7c0cd080 2023-06-08T03:51:11.972 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.968+0000 7fda637fe700 1 -- 172.21.15.79:0/3026057799 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda6c057bf0 con 0x7fda7c0cd080 2023-06-08T03:51:11.972 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fda6806ace0 msgr2=0x7fda6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fda6806ace0 0x7fda6806d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda70008030 comp rx=0 tx=0).stop 2023-06-08T03:51:11.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 msgr2=0x7fda7c0b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 0x7fda7c0b3f90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00e7e0 tx=0x7fda6c00eba0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 shutdown_connections 2023-06-08T03:51:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c0d0060 0x7fda7c0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fda6806ace0 0x7fda6806d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c0b20f0 0x7fda7c1e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 --2- 172.21.15.79:0/3026057799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda7c0cd080 0x7fda7c0b3f90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 >> 172.21.15.79:0/3026057799 conn(0x7fda7c06e1e0 msgr2=0x7fda7c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 shutdown_connections 2023-06-08T03:51:11.977 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:11.970+0000 7fda81047700 1 -- 172.21.15.79:0/3026057799 wait complete. 2023-06-08T03:51:11.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:12.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.105+0000 7fdf3e143700 1 Processor -- start 2023-06-08T03:51:12.109 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.105+0000 7fdf3e143700 1 -- start start 2023-06-08T03:51:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf3e143700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf38071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf3e143700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf38069d50 0x7fdf3806a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf3e143700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf380cd140 con 0x7fdf3815c1f0 2023-06-08T03:51:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf3e143700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3806a730 con 0x7fdf38069d50 2023-06-08T03:51:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf3e143700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38071110 con 0x7fdf3806d8a0 2023-06-08T03:51:12.111 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf38069d50 0x7fdf3806a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdf38069d50 0x7fdf3806a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36100/0 (socket says 172.21.15.79:36100) 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 -- 172.21.15.79:0/465674989 learned_addr learned my addr 172.21.15.79:0/465674989 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf36ffd700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf38071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 -- 172.21.15.79:0/465674989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 msgr2=0x7fdf38071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf38071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 -- 172.21.15.79:0/465674989 >> v1:172.21.15.156:6789/0 conn(0x7fdf3815c1f0 legacy=0x7fdf380cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf37fff700 1 -- 172.21.15.79:0/465674989 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38071be0 con 0x7fdf38069d50 2023-06-08T03:51:12.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.106+0000 7fdf36ffd700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf38071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:12.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf37fff700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf3806a1f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009e50 tx=0x7fdf3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651ad4a37492d1b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:12.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf367fc700 1 -- 172.21.15.79:0/465674989 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000e070 con 0x7fdf38069d50 2023-06-08T03:51:12.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf367fc700 1 -- 172.21.15.79:0/465674989 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30003a20 con 0x7fdf38069d50 2023-06-08T03:51:12.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf367fc700 1 -- 172.21.15.79:0/465674989 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003dd0 con 0x7fdf38069d50 2023-06-08T03:51:12.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 -- 172.21.15.79:0/465674989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 msgr2=0x7fdf3806a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf3806a1f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009e50 tx=0x7fdf3000b330 comp rx=0 tx=0).stop 2023-06-08T03:51:12.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 -- 172.21.15.79:0/465674989 shutdown_connections 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf3806a1f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 --2- 172.21.15.79:0/465674989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf38071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 -- 172.21.15.79:0/465674989 >> 172.21.15.79:0/465674989 conn(0x7fdf380b1d30 msgr2=0x7fdf3806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 -- 172.21.15.79:0/465674989 shutdown_connections 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.107+0000 7fdf3e143700 1 -- 172.21.15.79:0/465674989 wait complete. 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.109+0000 7fdf3e143700 1 Processor -- start 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.109+0000 7fdf3e143700 1 -- start start 2023-06-08T03:51:12.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf380c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf380c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3815c1f0 0x7fdf3815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c3ad0 con 0x7fdf38069d50 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c9440 con 0x7fdf3815c1f0 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c2e50 con 0x7fdf3806d8a0 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf380c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.117 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf380c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36102/0 (socket says 172.21.15.79:36102) 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 -- 172.21.15.79:0/1219516503 learned_addr learned my addr 172.21.15.79:0/1219516503 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf36ffd700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf380c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf37fff700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3815c1f0 0x7fdf3815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 -- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 msgr2=0x7fdf380c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf380c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 -- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3815c1f0 msgr2=0x7fdf3815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3815c1f0 0x7fdf3815e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 -- 172.21.15.79:0/1219516503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30009b30 con 0x7fdf38069d50 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf37fff700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3815c1f0 0x7fdf3815e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf377fe700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf380c3ef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00b4a0 tx=0x7fdf2c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011830 con 0x7fdf38069d50 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c011990 con 0x7fdf38069d50 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380c3130 con 0x7fdf38069d50 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c011d40 con 0x7fdf38069d50 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.110+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf380c36f0 con 0x7fdf38069d50 2023-06-08T03:51:12.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.111+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf380c3c10 con 0x7fdf38069d50 2023-06-08T03:51:12.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.112+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf2c01a400 con 0x7fdf38069d50 2023-06-08T03:51:12.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.112+0000 7fdf1ffff700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf2406ace0 0x7fdf2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.112+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf2c089460 con 0x7fdf38069d50 2023-06-08T03:51:12.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.112+0000 7fdf36ffd700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf2406ace0 0x7fdf2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.113+0000 7fdf36ffd700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf2406ace0 0x7fdf2406d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdf28006fd0 tx=0x7fdf28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:12.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.116+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf2c0542b0 con 0x7fdf38069d50 2023-06-08T03:51:12.308 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.305+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdf38073e90 con 0x7fdf38069d50 2023-06-08T03:51:12.309 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.305+0000 7fdf1ffff700 1 -- 172.21.15.79:0/1219516503 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdf2c0578d0 con 0x7fdf38069d50 2023-06-08T03:51:12.309 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:12.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf2406ace0 msgr2=0x7fdf2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf2406ace0 0x7fdf2406d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdf28006fd0 tx=0x7fdf28008040 comp rx=0 tx=0).stop 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 msgr2=0x7fdf380c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf380c3ef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00b4a0 tx=0x7fdf2c00b860 comp rx=0 tx=0).stop 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 shutdown_connections 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf3815c1f0 0x7fdf3815e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf38069d50 0x7fdf380c3ef0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf2406ace0 0x7fdf2406d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.312 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 --2- 172.21.15.79:0/1219516503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf3806d8a0 0x7fdf380c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 >> 172.21.15.79:0/1219516503 conn(0x7fdf380b1d30 msgr2=0x7fdf380b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:12.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 shutdown_connections 2023-06-08T03:51:12.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.307+0000 7fdf3e143700 1 -- 172.21.15.79:0/1219516503 wait complete. 2023-06-08T03:51:12.332 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-06-08T03:51:12.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-06-08T03:51:12.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.449+0000 7fb05126e700 1 Processor -- start 2023-06-08T03:51:12.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.449+0000 7fb05126e700 1 -- start start 2023-06-08T03:51:12.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb05126e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb04c15c150 0x7fb04c0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb05126e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c06a620 con 0x7fb04c15c520 2023-06-08T03:51:12.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb05126e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c06aea0 con 0x7fb04c06c620 2023-06-08T03:51:12.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb05126e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c06b1f0 con 0x7fb04c15c150 2023-06-08T03:51:12.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb04a7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb04c15c520 0x7fb04c15fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59586/0 (socket says 172.21.15.79:59586) 2023-06-08T03:51:12.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb04a7fc700 1 -- 172.21.15.79:0/3527781440 learned_addr learned my addr 172.21.15.79:0/3527781440 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226715630 0 0) 0x7fb03c001110 con 0x7fb04c15c520 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0340035b0 con 0x7fb04c15c520 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb04affd700 1 --2- 172.21.15.79:0/3527781440 >> v2:172.21.15.156:3300/0 conn(0x7fb04c15c150 0x7fb04c0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126293402 0 0) 0x7fb0340035b0 con 0x7fb04c15c520 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 >> v2:172.21.15.156:3300/0 conn(0x7fb04c15c150 msgr2=0x7fb04c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 --2- 172.21.15.79:0/3527781440 >> v2:172.21.15.156:3300/0 conn(0x7fb04c15c150 0x7fb04c0d0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 >> v1:172.21.15.156:6789/0 conn(0x7fb04c06c620 legacy=0x7fb04c160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0b1c70 con 0x7fb04c15c520 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fb03c001800 con 0x7fb04c15c520 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 >> v1:172.21.15.79:6789/0 conn(0x7fb04c15c520 legacy=0x7fb04c15fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.450+0000 7fb049ffb700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb034003a70 0x7fb034003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb034004850 0x7fb034004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034005200 0x7fb03400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c06b1f0 con 0x7fb034004850 2023-06-08T03:51:12.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03400c0e0 con 0x7fb034005200 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03400c2a0 con 0x7fb034003a70 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb034004850 0x7fb034004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04affd700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb034003a70 0x7fb034003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04b7fe700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034005200 0x7fb03400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 -- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb034003a70 msgr2=0x7fb034003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb034003a70 0x7fb034003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 -- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034005200 msgr2=0x7fb03400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:12.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034005200 0x7fb03400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 -- 172.21.15.79:0/3527781440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340035b0 con 0x7fb034004850 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04b7fe700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034005200 0x7fb03400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04affd700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb034003a70 0x7fb034003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04b7fe700 1 -- 172.21.15.79:0/3527781440 reap_dead start 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb04a7fc700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb034004850 0x7fb034004cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c240 tx=0x7fb034005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ce348c8082ced8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c007d80 con 0x7fb034004850 2023-06-08T03:51:12.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c00cda0 con 0x7fb034004850 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.451+0000 7fb049ffb700 1 -- 172.21.15.79:0/3527781440 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c005ba0 con 0x7fb034004850 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 -- 172.21.15.79:0/3527781440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb034004850 msgr2=0x7fb034004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb034004850 0x7fb034004cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c240 tx=0x7fb034005880 comp rx=0 tx=0).stop 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 -- 172.21.15.79:0/3527781440 shutdown_connections 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 --2- 172.21.15.79:0/3527781440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb034004850 0x7fb034004cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 -- 172.21.15.79:0/3527781440 >> 172.21.15.79:0/3527781440 conn(0x7fb04c09bbd0 msgr2=0x7fb04c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 -- 172.21.15.79:0/3527781440 shutdown_connections 2023-06-08T03:51:12.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.452+0000 7fb05126e700 1 -- 172.21.15.79:0/3527781440 wait complete. 2023-06-08T03:51:12.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.453+0000 7fb05126e700 1 Processor -- start 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.453+0000 7fb05126e700 1 -- start start 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034004850 0x7fb04c0b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0c8450 0x7fb04c14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 0x7fb04c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0b7920 con 0x7fb04c0b58e0 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0b7ee0 con 0x7fb034004850 2023-06-08T03:51:12.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c14b260 con 0x7fb04c0c8450 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 0x7fb04c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 0x7fb04c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36126/0 (socket says 172.21.15.79:36126) 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 -- 172.21.15.79:0/1154475212 learned_addr learned my addr 172.21.15.79:0/1154475212 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04a7fc700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0c8450 0x7fb04c14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04affd700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034004850 0x7fb04c0b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 -- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0c8450 msgr2=0x7fb04c14ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0c8450 0x7fb04c14ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 -- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034004850 msgr2=0x7fb04c0b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034004850 0x7fb04c0b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 -- 172.21.15.79:0/1154475212 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0b62c0 con 0x7fb04c0b58e0 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb04b7fe700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 0x7fb04c15ea30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb04c08bd00 tx=0x7fb04400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044003c10 con 0x7fb04c0b58e0 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb044003d70 con 0x7fb04c0b58e0 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0b65a0 con 0x7fb04c0b58e0 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.454+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0440052d0 con 0x7fb04c0b58e0 2023-06-08T03:51:12.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.455+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c0b5d50 con 0x7fb04c0b58e0 2023-06-08T03:51:12.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.455+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c073e90 con 0x7fb04c0b58e0 2023-06-08T03:51:12.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.456+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb044005430 con 0x7fb04c0b58e0 2023-06-08T03:51:12.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.456+0000 7fb0337fe700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb03806ace0 0x7fb03806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:12.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.456+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb044089860 con 0x7fb04c0b58e0 2023-06-08T03:51:12.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.456+0000 7fb04affd700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb03806ace0 0x7fb03806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:12.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.457+0000 7fb04affd700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb03806ace0 0x7fb03806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb04c04fa30 tx=0x7fb040008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:12.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.460+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb044054610 con 0x7fb04c0b58e0 2023-06-08T03:51:12.729 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:12.726+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7fb04c06a970 con 0x7fb04c0b58e0 2023-06-08T03:51:13.104 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.101+0000 7fb0337fe700 1 -- 172.21.15.79:0/1154475212 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v65) v1 ==== 147+0+0 (secure 0 0 0) 0x7fb044057c30 con 0x7fb04c0b58e0 2023-06-08T03:51:13.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb03806ace0 msgr2=0x7fb03806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb03806ace0 0x7fb03806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb04c04fa30 tx=0x7fb040008040 comp rx=0 tx=0).stop 2023-06-08T03:51:13.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 msgr2=0x7fb04c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.107 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 0x7fb04c15ea30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb04c08bd00 tx=0x7fb04400ee00 comp rx=0 tx=0).stop 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 shutdown_connections 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb04c0b58e0 0x7fb04c15ea30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb034004850 0x7fb04c0b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb03806ace0 0x7fb03806d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 --2- 172.21.15.79:0/1154475212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb04c0c8450 0x7fb04c14ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 >> 172.21.15.79:0/1154475212 conn(0x7fb04c09bbd0 msgr2=0x7fb04c0b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 shutdown_connections 2023-06-08T03:51:13.108 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.103+0000 7fb05126e700 1 -- 172.21.15.79:0/1154475212 wait complete. 2023-06-08T03:51:13.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-06-08T03:51:13.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.240+0000 7f23cd4f1700 1 Processor -- start 2023-06-08T03:51:13.244 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.240+0000 7f23cd4f1700 1 -- start start 2023-06-08T03:51:13.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23cd4f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f23c815ffa0 0x7f23c8160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23cd4f1700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c80b29a0 0x7f23c81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23cd4f1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c806ae60 con 0x7f23c806c750 2023-06-08T03:51:13.246 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23cd4f1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c815d430 con 0x7f23c815ffa0 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23cd4f1700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c815d710 con 0x7f23c80b29a0 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f23c815ffa0 0x7f23c8160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f23c815ffa0 0x7f23c8160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36130/0 (socket says 172.21.15.79:36130) 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 -- 172.21.15.79:0/430202687 learned_addr learned my addr 172.21.15.79:0/430202687 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c67fc700 1 --2- 172.21.15.79:0/430202687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c80b29a0 0x7f23c81608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 -- 172.21.15.79:0/430202687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c80b29a0 msgr2=0x7f23c81608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 --2- 172.21.15.79:0/430202687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c80b29a0 0x7f23c81608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.247 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 -- 172.21.15.79:0/430202687 >> v1:172.21.15.156:6789/0 conn(0x7f23c806c750 legacy=0x7f23c80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 -- 172.21.15.79:0/430202687 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80ce5c0 con 0x7f23c815ffa0 2023-06-08T03:51:13.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.241+0000 7f23c6ffd700 1 --2- 172.21.15.79:0/430202687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c815ffa0 0x7f23c8160370 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098f0 tx=0x7f23bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c27e069cb70d7b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:13.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23c5ffb700 1 -- 172.21.15.79:0/430202687 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc011830 con 0x7f23c815ffa0 2023-06-08T03:51:13.248 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23c5ffb700 1 -- 172.21.15.79:0/430202687 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc011990 con 0x7f23c815ffa0 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23c5ffb700 1 -- 172.21.15.79:0/430202687 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc011d20 con 0x7f23c815ffa0 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 -- 172.21.15.79:0/430202687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c815ffa0 msgr2=0x7f23c8160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/430202687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c815ffa0 0x7f23c8160370 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098f0 tx=0x7f23bc009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 -- 172.21.15.79:0/430202687 shutdown_connections 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/430202687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c815ffa0 0x7f23c8160370 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/430202687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c80b29a0 0x7f23c81608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.249 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 -- 172.21.15.79:0/430202687 >> 172.21.15.79:0/430202687 conn(0x7f23c8069d50 msgr2=0x7f23c806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:13.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 -- 172.21.15.79:0/430202687 shutdown_connections 2023-06-08T03:51:13.250 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.242+0000 7f23cd4f1700 1 -- 172.21.15.79:0/430202687 wait complete. 2023-06-08T03:51:13.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 Processor -- start 2023-06-08T03:51:13.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 -- start start 2023-06-08T03:51:13.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c806c750 0x7f23c81e6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c815ffa0 0x7f23c8146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 0x7f23c815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c81474a0 con 0x7f23c8147000 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c815fa70 con 0x7f23c806c750 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23cd4f1700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c815f790 con 0x7f23c815ffa0 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 0x7f23c815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23c77fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 0x7f23c815ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36140/0 (socket says 172.21.15.79:36140) 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.244+0000 7f23c77fe700 1 -- 172.21.15.79:0/1035045068 learned_addr learned my addr 172.21.15.79:0/1035045068 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c67fc700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c815ffa0 0x7f23c8146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.253 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c6ffd700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c806c750 0x7f23c81e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c77fe700 1 -- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c815ffa0 msgr2=0x7f23c8146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c77fe700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c815ffa0 0x7f23c8146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c77fe700 1 -- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c806c750 msgr2=0x7f23c81e6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c77fe700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c806c750 0x7f23c81e6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c77fe700 1 -- 172.21.15.79:0/1035045068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc009550 con 0x7f23c8147000 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23c77fe700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 0x7f23c815ea60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23c000cc50 tx=0x7f23c000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0003c00 con 0x7f23c8147000 2023-06-08T03:51:13.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0003d60 con 0x7f23c8147000 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c80b47f0 con 0x7f23c8147000 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c000aab0 con 0x7f23c8147000 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.245+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c80b4db0 con 0x7f23c8147000 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.246+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c80cf080 con 0x7f23c8147000 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.247+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23c000ac10 con 0x7f23c8147000 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.247+0000 7f23af7fe700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23b406ace0 0x7f23b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.255 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.247+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f23c0089620 con 0x7f23c8147000 2023-06-08T03:51:13.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.247+0000 7f23c6ffd700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23b406ace0 0x7f23b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.248+0000 7f23c6ffd700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23b406ace0 0x7f23b406d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23bc014040 tx=0x7f23bc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:13.256 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.251+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23c0054470 con 0x7f23c8147000 2023-06-08T03:51:13.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.440+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f23c8073e90 con 0x7f23c8147000 2023-06-08T03:51:13.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.446+0000 7f23af7fe700 1 -- 172.21.15.79:0/1035045068 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f23c0057a90 con 0x7f23c8147000 2023-06-08T03:51:13.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23b406ace0 msgr2=0x7f23b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23b406ace0 0x7f23b406d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23bc014040 tx=0x7f23bc008030 comp rx=0 tx=0).stop 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 msgr2=0x7f23c815ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 0x7f23c815ea60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23c000cc50 tx=0x7f23c000cf60 comp rx=0 tx=0).stop 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 shutdown_connections 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23c8147000 0x7f23c815ea60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c806c750 0x7f23c81e6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f23b406ace0 0x7f23b406d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 --2- 172.21.15.79:0/1035045068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c815ffa0 0x7f23c8146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.448+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 >> 172.21.15.79:0/1035045068 conn(0x7f23c8069d50 msgr2=0x7f23c806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:13.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.449+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 shutdown_connections 2023-06-08T03:51:13.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.449+0000 7f23cd4f1700 1 -- 172.21.15.79:0/1035045068 wait complete. 2023-06-08T03:51:13.464 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-08T03:51:13.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-08T03:51:13.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-08T03:51:13.555 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-08T03:51:13.555 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:51:13.555 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:13.572 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:51:13.573 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:13.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:13.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.789+0000 7fe99bda0700 1 Processor -- start 2023-06-08T03:51:13.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.789+0000 7fe99bda0700 1 -- start start 2023-06-08T03:51:13.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe99bda0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe994159270 0x7fe9940cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe99bda0700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99406ae60 con 0x7fe9940ca630 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe99bda0700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99406b9e0 con 0x7fe9940b16f0 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe99bda0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9940cebc0 con 0x7fe994159270 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe99933b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe9940ca630 0x7fe9940b0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59624/0 (socket says 172.21.15.79:59624) 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe99933b700 1 -- 172.21.15.79:0/52688644 learned_addr learned my addr 172.21.15.79:0/52688644 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943871683 0 0) 0x7fe99406ae60 con 0x7fe9940ca630 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9800035b0 con 0x7fe9940ca630 2023-06-08T03:51:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.790+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 861914580 0 0) 0x7fe9800035b0 con 0x7fe9940ca630 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 >> v2:172.21.15.156:3300/0 conn(0x7fe994159270 msgr2=0x7fe9940cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 --2- 172.21.15.79:0/52688644 >> v2:172.21.15.156:3300/0 conn(0x7fe994159270 0x7fe9940cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 >> v1:172.21.15.156:6789/0 conn(0x7fe9940b16f0 legacy=0x7fe99415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99406b0b0 con 0x7fe9940ca630 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fe990001670 con 0x7fe9940ca630 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 >> v1:172.21.15.79:6789/0 conn(0x7fe9940ca630 legacy=0x7fe9940b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980003a70 0x7fe980003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe980004850 0x7fe980004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980005400 0x7fe98000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9940cebc0 con 0x7fe980004850 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98000c2f0 con 0x7fe980005400 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98000e6c0 con 0x7fe980003a70 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe980004850 0x7fe980004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99a33d700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980005400 0x7fe98000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe999b3c700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980003a70 0x7fe980003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 -- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980003a70 msgr2=0x7fe980003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980003a70 0x7fe980003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 -- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980005400 msgr2=0x7fe98000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe980005400 0x7fe98000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99a33d700 1 -- 172.21.15.79:0/52688644 reap_dead start 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 -- 172.21.15.79:0/52688644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800035b0 con 0x7fe980004850 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe99933b700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe980004850 0x7fe980004cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9940caaa0 tx=0x7fe980005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe92cd023597a4e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99000ce00 con 0x7fe980004850 2023-06-08T03:51:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990005960 con 0x7fe980004850 2023-06-08T03:51:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.791+0000 7fe998b3a700 1 -- 172.21.15.79:0/52688644 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990005d10 con 0x7fe980004850 2023-06-08T03:51:13.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 -- 172.21.15.79:0/52688644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe980004850 msgr2=0x7fe980004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe980004850 0x7fe980004cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9940caaa0 tx=0x7fe980005880 comp rx=0 tx=0).stop 2023-06-08T03:51:13.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 -- 172.21.15.79:0/52688644 shutdown_connections 2023-06-08T03:51:13.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 --2- 172.21.15.79:0/52688644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe980004850 0x7fe980004cc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 -- 172.21.15.79:0/52688644 >> 172.21.15.79:0/52688644 conn(0x7fe9940cbfe0 msgr2=0x7fe9940cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 -- 172.21.15.79:0/52688644 shutdown_connections 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.792+0000 7fe99bda0700 1 -- 172.21.15.79:0/52688644 wait complete. 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.793+0000 7fe99bda0700 1 Processor -- start 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 -- start start 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980004850 0x7fe9941f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 0x7fe9940b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9940b3b00 0x7fe99415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941f6550 con 0x7fe9940acde0 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941f3590 con 0x7fe9940b3b00 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99bda0700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9941f5c20 con 0x7fe980004850 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99933b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 0x7fe9940b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99933b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 0x7fe9940b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36158/0 (socket says 172.21.15.79:36158) 2023-06-08T03:51:13.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.794+0000 7fe99933b700 1 -- 172.21.15.79:0/2726415580 learned_addr learned my addr 172.21.15.79:0/2726415580 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99a33d700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9940b3b00 0x7fe99415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99933b700 1 -- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980004850 msgr2=0x7fe9941f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99933b700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980004850 0x7fe9941f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99933b700 1 -- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9940b3b00 msgr2=0x7fe99415aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99933b700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9940b3b00 0x7fe99415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99933b700 1 -- 172.21.15.79:0/2726415580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9941f5f00 con 0x7fe9940acde0 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99933b700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 0x7fe9940b35c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe990001930 tx=0x7fe990001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:13.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe990004a80 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe990004be0 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900070d0 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9941f61e0 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.795+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9940ad3a0 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.796+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9940ad5c0 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.796+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe990007230 con 0x7fe9940acde0 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.797+0000 7fe98a7fc700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:13.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.797+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe99008aa20 con 0x7fe9940acde0 2023-06-08T03:51:13.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.797+0000 7fe999b3c700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:13.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.797+0000 7fe999b3c700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe97806ace0 0x7fe97806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe984006fd0 tx=0x7fe984008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:13.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.801+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe990055930 con 0x7fe9940acde0 2023-06-08T03:51:13.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.989+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9941f5d60 con 0x7fe9940acde0 2023-06-08T03:51:13.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.990+0000 7fe98a7fc700 1 -- 172.21.15.79:0/2726415580 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fe9941f5d60 con 0x7fe9940acde0 2023-06-08T03:51:13.995 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:13.995 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:13.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe97806ace0 msgr2=0x7fe97806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe97806ace0 0x7fe97806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe984006fd0 tx=0x7fe984008040 comp rx=0 tx=0).stop 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 msgr2=0x7fe9940b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 0x7fe9940b35c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe990001930 tx=0x7fe990001a00 comp rx=0 tx=0).stop 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 shutdown_connections 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9940b3b00 0x7fe99415aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe97806ace0 0x7fe97806d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe980004850 0x7fe9941f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 --2- 172.21.15.79:0/2726415580 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9940acde0 0x7fe9940b35c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:13.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 >> 172.21.15.79:0/2726415580 conn(0x7fe9940cbfe0 msgr2=0x7fe9940ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:13.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 shutdown_connections 2023-06-08T03:51:13.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:13.992+0000 7fe99bda0700 1 -- 172.21.15.79:0/2726415580 wait complete. 2023-06-08T03:51:14.009 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:14.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:14.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.130+0000 7f0ce83e8700 1 Processor -- start 2023-06-08T03:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.130+0000 7f0ce83e8700 1 -- start start 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce83e8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0ce015c0a0 0x7f0ce0070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce83e8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0ce00cd590 0x7f0ce00cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce83e8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce0070bd0 con 0x7f0ce015fb10 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce83e8700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce015c470 con 0x7f0ce00cd590 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce83e8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce00cacc0 con 0x7f0ce015c0a0 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce6985700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0ce00cd590 0x7f0ce00cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce6985700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0ce00cd590 0x7f0ce00cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36168/0 (socket says 172.21.15.79:36168) 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.131+0000 7f0ce6985700 1 -- 172.21.15.79:0/2204601516 learned_addr learned my addr 172.21.15.79:0/2204601516 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce6985700 1 -- 172.21.15.79:0/2204601516 >> v2:172.21.15.156:3300/0 conn(0x7f0ce015c0a0 msgr2=0x7f0ce0070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce6985700 1 --2- 172.21.15.79:0/2204601516 >> v2:172.21.15.156:3300/0 conn(0x7f0ce015c0a0 0x7f0ce0070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce6985700 1 -- 172.21.15.79:0/2204601516 >> v1:172.21.15.156:6789/0 conn(0x7f0ce015fb10 legacy=0x7f0ce00cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce6985700 1 -- 172.21.15.79:0/2204601516 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00ce560 con 0x7f0ce00cd590 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce6985700 1 --2- 172.21.15.79:0/2204601516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00cb110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006ab40 tx=0x7f0cd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38d5aa9dc450f9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce5182700 1 -- 172.21.15.79:0/2204601516 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800e070 con 0x7f0ce00cd590 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce5182700 1 -- 172.21.15.79:0/2204601516 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8003a20 con 0x7f0ce00cd590 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce5182700 1 -- 172.21.15.79:0/2204601516 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8003dd0 con 0x7f0ce00cd590 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2204601516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 msgr2=0x7f0ce00cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.132+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2204601516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00cb110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006ab40 tx=0x7f0cd800b330 comp rx=0 tx=0).stop 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.133+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2204601516 shutdown_connections 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.133+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2204601516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00cb110 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.133+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2204601516 >> v2:172.21.15.156:3300/0 conn(0x7f0ce015c0a0 0x7f0ce0070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.133+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2204601516 >> 172.21.15.79:0/2204601516 conn(0x7f0ce006e1e0 msgr2=0x7f0ce006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.133+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2204601516 shutdown_connections 2023-06-08T03:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.133+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2204601516 wait complete. 2023-06-08T03:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.134+0000 7f0ce83e8700 1 Processor -- start 2023-06-08T03:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.134+0000 7f0ce83e8700 1 -- start start 2023-06-08T03:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce015c0a0 0x7f0ce0152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce015fb10 0x7f0ce015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c67d0 con 0x7f0ce00cd590 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c5bc0 con 0x7f0ce015c0a0 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c5ea0 con 0x7f0ce015fb10 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36170/0 (socket says 172.21.15.79:36170) 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 -- 172.21.15.79:0/2642186987 learned_addr learned my addr 172.21.15.79:0/2642186987 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6985700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce015fb10 0x7f0ce015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce5983700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce015c0a0 0x7f0ce0152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 -- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce015fb10 msgr2=0x7f0ce015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce015fb10 0x7f0ce015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 -- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce015c0a0 msgr2=0x7f0ce0152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce015c0a0 0x7f0ce0152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 -- 172.21.15.79:0/2642186987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8009b30 con 0x7f0ce00cd590 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce5983700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce015c0a0 0x7f0ce0152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce6184700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00c6c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000b4a0 tx=0x7f0cd000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0011830 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0011990 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce00c6180 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0011d40 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.135+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0153490 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.136+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce00c6ab0 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.137+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0cd001a400 con 0x7f0ce00cd590 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.137+0000 7f0cd6ffd700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0cc406ad10 0x7f0cc406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.137+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0cd00896c0 con 0x7f0ce00cd590 2023-06-08T03:51:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.137+0000 7f0ce5983700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0cc406ad10 0x7f0cc406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.138+0000 7f0ce5983700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0cc406ad10 0x7f0cc406d1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00cca80 tx=0x7f0cdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.146 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.141+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0cd0054490 con 0x7f0ce00cd590 2023-06-08T03:51:14.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.327+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ce0073e90 con 0x7f0ce00cd590 2023-06-08T03:51:14.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.327+0000 7f0cd6ffd700 1 -- 172.21.15.79:0/2642186987 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0cd0057ab0 con 0x7f0ce00cd590 2023-06-08T03:51:14.332 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:14.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0cc406ad10 msgr2=0x7f0cc406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0cc406ad10 0x7f0cc406d1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00cca80 tx=0x7f0cdc009040 comp rx=0 tx=0).stop 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 msgr2=0x7f0ce00c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00c6c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000b4a0 tx=0x7f0cd000b860 comp rx=0 tx=0).stop 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 shutdown_connections 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0cc406ad10 0x7f0cc406d1c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0ce015fb10 0x7f0ce015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ce00cd590 0x7f0ce00c6c60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 --2- 172.21.15.79:0/2642186987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ce015c0a0 0x7f0ce0152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 >> 172.21.15.79:0/2642186987 conn(0x7f0ce006e1e0 msgr2=0x7f0ce00cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 shutdown_connections 2023-06-08T03:51:14.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.329+0000 7f0ce83e8700 1 -- 172.21.15.79:0/2642186987 wait complete. 2023-06-08T03:51:14.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:14.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.464+0000 7f1f67c23700 1 Processor -- start 2023-06-08T03:51:14.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.464+0000 7f1f67c23700 1 -- start start 2023-06-08T03:51:14.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.465+0000 7f1f67c23700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 0x7f1f60163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.465+0000 7f1f67c23700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f600cd7b0 con 0x7f1f600ca360 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.465+0000 7f1f67c23700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f600cda50 con 0x7f1f60158d50 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.465+0000 7f1f67c23700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600cdd30 con 0x7f1f600cc690 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.465+0000 7f1f651be700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1f600ca360 0x7f1f600cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59654/0 (socket says 172.21.15.79:59654) 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.465+0000 7f1f651be700 1 -- 172.21.15.79:0/556096035 learned_addr learned my addr 172.21.15.79:0/556096035 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430990655 0 0) 0x7f1f600cd7b0 con 0x7f1f600ca360 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f4c0035b0 con 0x7f1f600ca360 2023-06-08T03:51:14.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f661c0700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 0x7f1f60163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659586527 0 0) 0x7f1f4c0035b0 con 0x7f1f600ca360 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 msgr2=0x7f1f60163d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 0x7f1f60163d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 >> v1:172.21.15.156:6789/0 conn(0x7f1f60158d50 legacy=0x7f1f6015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600ce010 con 0x7f1f600ca360 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f1f5c001670 con 0x7f1f600ca360 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 >> v1:172.21.15.79:6789/0 conn(0x7f1f600ca360 legacy=0x7f1f600cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f4c003a30 0x7f1f4c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 0x7f1f4c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f4c004d90 0x7f1f4c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600cdd30 con 0x7f1f4c003a30 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00bac0 con 0x7f1f4c004d90 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00de60 con 0x7f1f600cc690 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f4c003a30 0x7f1f4c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f661c0700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f4c004d90 0x7f1f4c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f651be700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 0x7f1f4c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 -- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 msgr2=0x7f1f4c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f600cc690 0x7f1f4c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 -- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f4c004d90 msgr2=0x7f1f4c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f4c004d90 0x7f1f4c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 -- 172.21.15.79:0/556096035 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0035b0 con 0x7f1f4c003a30 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.466+0000 7f1f659bf700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f4c003a30 0x7f1f4c003e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1f50001800 tx=0x7f1f4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5306f80c9125766 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f500012f0 con 0x7f1f4c003a30 2023-06-08T03:51:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f500052f0 con 0x7f1f4c003a30 2023-06-08T03:51:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f649bd700 1 -- 172.21.15.79:0/556096035 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50006860 con 0x7f1f4c003a30 2023-06-08T03:51:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f67c23700 1 -- 172.21.15.79:0/556096035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f4c003a30 msgr2=0x7f1f4c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f67c23700 1 --2- 172.21.15.79:0/556096035 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f4c003a30 0x7f1f4c003e40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1f50001800 tx=0x7f1f4c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f661c0700 1 -- 172.21.15.79:0/556096035 reap_dead start 2023-06-08T03:51:14.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f67c23700 1 -- 172.21.15.79:0/556096035 shutdown_connections 2023-06-08T03:51:14.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f67c23700 1 -- 172.21.15.79:0/556096035 >> 172.21.15.79:0/556096035 conn(0x7f1f600b1d30 msgr2=0x7f1f6006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:14.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f67c23700 1 -- 172.21.15.79:0/556096035 shutdown_connections 2023-06-08T03:51:14.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.467+0000 7f1f67c23700 1 -- 172.21.15.79:0/556096035 wait complete. 2023-06-08T03:51:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.469+0000 7f1f67c23700 1 Processor -- start 2023-06-08T03:51:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.469+0000 7f1f67c23700 1 -- start start 2023-06-08T03:51:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c003a30 0x7f1f601ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f60155000 0x7f1f60155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 0x7f1f600c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600b2f90 con 0x7f1f600c4e70 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f601559b0 con 0x7f1f60155000 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600b23a0 con 0x7f1f4c003a30 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 0x7f1f600c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 0x7f1f600c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36194/0 (socket says 172.21.15.79:36194) 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 -- 172.21.15.79:0/2547656129 learned_addr learned my addr 172.21.15.79:0/2547656129 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:14.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f651be700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f60155000 0x7f1f60155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f659bf700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c003a30 0x7f1f601ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 -- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c003a30 msgr2=0x7f1f601ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c003a30 0x7f1f601ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 -- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f60155000 msgr2=0x7f1f60155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f60155000 0x7f1f60155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 -- 172.21.15.79:0/2547656129 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600b33b0 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f661c0700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 0x7f1f600c52f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1f6006fbe0 tx=0x7f1f580111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800e070 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f58003cf0 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58003ea0 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f600b25f0 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.470+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f600b2bb0 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.471+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f600b30d0 con 0x7f1f600c4e70 2023-06-08T03:51:14.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.472+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f580117a0 con 0x7f1f600c4e70 2023-06-08T03:51:14.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.472+0000 7f1f567fc700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f4406ace0 0x7f1f4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.472+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f580898a0 con 0x7f1f600c4e70 2023-06-08T03:51:14.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.472+0000 7f1f659bf700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f4406ace0 0x7f1f4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.473+0000 7f1f659bf700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f4406ace0 0x7f1f4406d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f50014040 tx=0x7f1f50009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.476+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1f600b30d0 con 0x7f1f600c4e70 2023-06-08T03:51:14.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.663+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f600b30d0 con 0x7f1f600c4e70 2023-06-08T03:51:14.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.663+0000 7f1f567fc700 1 -- 172.21.15.79:0/2547656129 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f600b30d0 con 0x7f1f600c4e70 2023-06-08T03:51:14.668 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:14.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f4406ace0 msgr2=0x7f1f4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f4406ace0 0x7f1f4406d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f50014040 tx=0x7f1f50009030 comp rx=0 tx=0).stop 2023-06-08T03:51:14.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 msgr2=0x7f1f600c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 0x7f1f600c52f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1f6006fbe0 tx=0x7f1f580111d0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 shutdown_connections 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f600c4e70 0x7f1f600c52f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f4406ace0 0x7f1f4406d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c003a30 0x7f1f601ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 --2- 172.21.15.79:0/2547656129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f60155000 0x7f1f60155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 >> 172.21.15.79:0/2547656129 conn(0x7f1f600b1d30 msgr2=0x7f1f600cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:14.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 shutdown_connections 2023-06-08T03:51:14.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.665+0000 7f1f67c23700 1 -- 172.21.15.79:0/2547656129 wait complete. 2023-06-08T03:51:14.681 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:14.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.799+0000 7f41dca08700 1 Processor -- start 2023-06-08T03:51:14.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.799+0000 7f41dca08700 1 -- start start 2023-06-08T03:51:14.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41dca08700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f41d80cce80 0x7f41d8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41dca08700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d8070bd0 con 0x7f41d80cb1e0 2023-06-08T03:51:14.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41dca08700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d80cad50 con 0x7f41d815fb10 2023-06-08T03:51:14.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41dca08700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80ce570 con 0x7f41d80cce80 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d6d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f41d80cb1e0 0x7f41d80cb660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59676/0 (socket says 172.21.15.79:59676) 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d6d9d700 1 -- 172.21.15.79:0/2006676489 learned_addr learned my addr 172.21.15.79:0/2006676489 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318866555 0 0) 0x7f41d0001490 con 0x7f41d80cb1e0 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d659c700 1 --2- 172.21.15.79:0/2006676489 >> v2:172.21.15.156:3300/0 conn(0x7f41d80cce80 0x7f41d8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c00035b0 con 0x7f41d80cb1e0 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795276126 0 0) 0x7f41c00035b0 con 0x7f41d80cb1e0 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 >> v2:172.21.15.156:3300/0 conn(0x7f41d80cce80 msgr2=0x7f41d8070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 --2- 172.21.15.79:0/2006676489 >> v2:172.21.15.156:3300/0 conn(0x7f41d80cce80 0x7f41d8070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 >> v1:172.21.15.156:6789/0 conn(0x7f41d815fb10 legacy=0x7f41d80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d8160080 con 0x7f41d80cb1e0 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f41d0001b80 con 0x7f41d80cb1e0 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 >> v1:172.21.15.79:6789/0 conn(0x7f41d80cb1e0 legacy=0x7f41d80cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c0003a00 0x7f41c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41c0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c0005200 0x7f41c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80ce570 con 0x7f41c0004850 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c000c080 con 0x7f41c0003a00 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.800+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c000c240 con 0x7f41c0005200 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41c0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d659c700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c0003a00 0x7f41c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d6d9d700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c0005200 0x7f41c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 -- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c0005200 msgr2=0x7f41c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41c0005200 0x7f41c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 -- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c0003a00 msgr2=0x7f41c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41c0003a00 0x7f41c0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 -- 172.21.15.79:0/2006676489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c00035b0 con 0x7f41c0004850 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d6d9d700 1 -- 172.21.15.79:0/2006676489 reap_dead start 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41c0004cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f41d815ffb0 tx=0x7f41c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1fdcde593c6567 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0052d0 con 0x7f41c0004850 2023-06-08T03:51:14.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc005470 con 0x7f41c0004850 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.801+0000 7f41d559a700 1 -- 172.21.15.79:0/2006676489 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0038c0 con 0x7f41c0004850 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 -- 172.21.15.79:0/2006676489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 msgr2=0x7f41c0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41c0004cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f41d815ffb0 tx=0x7f41c0005880 comp rx=0 tx=0).stop 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 -- 172.21.15.79:0/2006676489 shutdown_connections 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 --2- 172.21.15.79:0/2006676489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41c0004cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 -- 172.21.15.79:0/2006676489 >> 172.21.15.79:0/2006676489 conn(0x7f41d806e1e0 msgr2=0x7f41d806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 -- 172.21.15.79:0/2006676489 shutdown_connections 2023-06-08T03:51:14.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.802+0000 7f41dca08700 1 -- 172.21.15.79:0/2006676489 wait complete. 2023-06-08T03:51:14.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.803+0000 7f41dca08700 1 Processor -- start 2023-06-08T03:51:14.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.803+0000 7f41dca08700 1 -- start start 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41dca08700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41d8155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41dca08700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41d80c6340 0x7f41d815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41dca08700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41d80b09b0 0x7f41d80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41dca08700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8155670 con 0x7f41c0004850 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41dca08700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d815c770 con 0x7f41d80c6340 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41dca08700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d80b1300 con 0x7f41d80b09b0 2023-06-08T03:51:14.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41d8155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41d8155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36210/0 (socket says 172.21.15.79:36210) 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 -- 172.21.15.79:0/3747846632 learned_addr learned my addr 172.21.15.79:0/3747846632 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41d80c6340 0x7f41d815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d6d9d700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41d80b09b0 0x7f41d80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 -- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41d80b09b0 msgr2=0x7f41d80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41d80b09b0 0x7f41d80b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 -- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41d80c6340 msgr2=0x7f41d815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:14.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41d80c6340 0x7f41d815c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 -- 172.21.15.79:0/3747846632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80b15e0 con 0x7f41c0004850 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.804+0000 7f41d659c700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41d8155130 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f41c8011aa0 tx=0x7f41c8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.805+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8011ee0 con 0x7f41c0004850 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.805+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c800f8c0 con 0x7f41c0004850 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.805+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c800fc90 con 0x7f41c0004850 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.805+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d80b18c0 con 0x7f41c0004850 2023-06-08T03:51:14.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.805+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d80b1e80 con 0x7f41c0004850 2023-06-08T03:51:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.806+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d80b2010 con 0x7f41c0004850 2023-06-08T03:51:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.806+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41c8014440 con 0x7f41c0004850 2023-06-08T03:51:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.806+0000 7f41c6ffd700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f41b806ace0 0x7f41b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.806+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f41c8089f40 con 0x7f41c0004850 2023-06-08T03:51:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.807+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f41b806ace0 0x7f41b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:14.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.807+0000 7f41d5d9b700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f41b806ace0 0x7f41b806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41d815ffb0 tx=0x7f41cc009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:14.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.811+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41c8054d40 con 0x7f41c0004850 2023-06-08T03:51:15.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.997+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41d8073e90 con 0x7f41c0004850 2023-06-08T03:51:15.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.997+0000 7f41c6ffd700 1 -- 172.21.15.79:0/3747846632 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41c8022020 con 0x7f41c0004850 2023-06-08T03:51:15.001 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:15.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f41b806ace0 msgr2=0x7f41b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f41b806ace0 0x7f41b806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41d815ffb0 tx=0x7f41cc009030 comp rx=0 tx=0).stop 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 msgr2=0x7f41d8155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41d8155130 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f41c8011aa0 tx=0x7f41c8011db0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 shutdown_connections 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f41b806ace0 0x7f41b806d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41d80b09b0 0x7f41d80b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41c0004850 0x7f41d8155130 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 --2- 172.21.15.79:0/3747846632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41d80c6340 0x7f41d815c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 >> 172.21.15.79:0/3747846632 conn(0x7f41d806e1e0 msgr2=0x7f41d8157160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:15.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 shutdown_connections 2023-06-08T03:51:15.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:14.999+0000 7f41dca08700 1 -- 172.21.15.79:0/3747846632 wait complete. 2023-06-08T03:51:15.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:15.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.135+0000 7fe562a3f700 1 Processor -- start 2023-06-08T03:51:15.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.135+0000 7fe562a3f700 1 -- start start 2023-06-08T03:51:15.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe562a3f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c15d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe562a3f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe55c071070 0x7fe55c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe562a3f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c15d8d0 con 0x7fe55c0cc480 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe562a3f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c06df90 con 0x7fe55c071070 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe562a3f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0cc8f0 con 0x7fe55c158ec0 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe55c071070 0x7fe55c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe55c071070 0x7fe55c0cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36218/0 (socket says 172.21.15.79:36218) 2023-06-08T03:51:15.142 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1153007931 learned_addr learned my addr 172.21.15.79:0/1153007931 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1153007931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 msgr2=0x7fe55c15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 --2- 172.21.15.79:0/1153007931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1153007931 >> v1:172.21.15.156:6789/0 conn(0x7fe55c0cc480 legacy=0x7fe55c06eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1153007931 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0ca320 con 0x7fe55c071070 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.136+0000 7fe55b7fe700 1 --2- 172.21.15.79:0/1153007931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c071070 0x7fe55c0cbf40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009920 tx=0x7fe54c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def9c536c4219c1b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe55affd700 1 -- 172.21.15.79:0/1153007931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004230 con 0x7fe55c071070 2023-06-08T03:51:15.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe55affd700 1 -- 172.21.15.79:0/1153007931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c004390 con 0x7fe55c071070 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe55affd700 1 -- 172.21.15.79:0/1153007931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c009e70 con 0x7fe55c071070 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 -- 172.21.15.79:0/1153007931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c071070 msgr2=0x7fe55c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1153007931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c071070 0x7fe55c0cbf40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009920 tx=0x7fe54c009c30 comp rx=0 tx=0).stop 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 -- 172.21.15.79:0/1153007931 shutdown_connections 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1153007931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c15d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1153007931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c071070 0x7fe55c0cbf40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.144 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 -- 172.21.15.79:0/1153007931 >> 172.21.15.79:0/1153007931 conn(0x7fe55c0b1d30 msgr2=0x7fe55c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 -- 172.21.15.79:0/1153007931 shutdown_connections 2023-06-08T03:51:15.145 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.137+0000 7fe562a3f700 1 -- 172.21.15.79:0/1153007931 wait complete. 2023-06-08T03:51:15.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.138+0000 7fe562a3f700 1 Processor -- start 2023-06-08T03:51:15.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 -- start start 2023-06-08T03:51:15.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c071070 0x7fe55c164110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 0x7fe55c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c161e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.147 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c149440 con 0x7fe55c0cc480 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c1647f0 con 0x7fe55c071070 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe562a3f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c148d80 con 0x7fe55c158ec0 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe55b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 0x7fe55c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe55b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 0x7fe55c0c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36226/0 (socket says 172.21.15.79:36226) 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1996875636 learned_addr learned my addr 172.21.15.79:0/1996875636 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe55bfff700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c071070 0x7fe55c164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.139+0000 7fe560fdc700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 msgr2=0x7fe55c161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55b7fe700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c071070 msgr2=0x7fe55c164110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55b7fe700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c071070 0x7fe55c164110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55b7fe700 1 -- 172.21.15.79:0/1996875636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009550 con 0x7fe55c0cc480 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe560fdc700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:15.149 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55bfff700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c071070 0x7fe55c164110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe55b7fe700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 0x7fe55c0c23b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009840 tx=0x7fe54c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00bd50 con 0x7fe55c0cc480 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c01f070 con 0x7fe55c0cc480 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c149060 con 0x7fe55c0cc480 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0038c0 con 0x7fe55c0cc480 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.140+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c0c2cf0 con 0x7fe55c0cc480 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.141+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c148be0 con 0x7fe55c0cc480 2023-06-08T03:51:15.150 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.141+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe54c021030 con 0x7fe55c0cc480 2023-06-08T03:51:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.142+0000 7fe558ff9700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe54806ace0 0x7fe54806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.142+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe54c08ad70 con 0x7fe55c0cc480 2023-06-08T03:51:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.142+0000 7fe55bfff700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe54806ace0 0x7fe54806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.142+0000 7fe55bfff700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe54806ace0 0x7fe54806d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe550006fd0 tx=0x7fe550008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.146+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe55c148be0 con 0x7fe55c0cc480 2023-06-08T03:51:15.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.331+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe55c149580 con 0x7fe55c0cc480 2023-06-08T03:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.332+0000 7fe558ff9700 1 -- 172.21.15.79:0/1996875636 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe55c149580 con 0x7fe55c0cc480 2023-06-08T03:51:15.336 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:15.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe54806ace0 msgr2=0x7fe54806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe54806ace0 0x7fe54806d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe550006fd0 tx=0x7fe550008040 comp rx=0 tx=0).stop 2023-06-08T03:51:15.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 msgr2=0x7fe55c0c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 0x7fe55c0c23b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009840 tx=0x7fe54c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:15.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 shutdown_connections 2023-06-08T03:51:15.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe54806ace0 0x7fe54806d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe55c158ec0 0x7fe55c161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe55c071070 0x7fe55c164110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 --2- 172.21.15.79:0/1996875636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe55c0cc480 0x7fe55c0c23b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 >> 172.21.15.79:0/1996875636 conn(0x7fe55c0b1d30 msgr2=0x7fe55c14b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 shutdown_connections 2023-06-08T03:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.334+0000 7fe562a3f700 1 -- 172.21.15.79:0/1996875636 wait complete. 2023-06-08T03:51:15.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:15.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.467+0000 7fa2c5745700 1 Processor -- start 2023-06-08T03:51:15.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.467+0000 7fa2c5745700 1 -- start start 2023-06-08T03:51:15.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2c5745700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa2c015c1f0 0x7fa2c006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2c5745700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa2c0071fe0 0x7fa2c00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2c5745700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c00cd140 con 0x7fa2c00cf130 2023-06-08T03:51:15.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2c5745700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c0071110 con 0x7fa2c015c1f0 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2c5745700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c0071be0 con 0x7fa2c0071fe0 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2beffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa2c015c1f0 0x7fa2c006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2bf7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa2c00cf130 0x7fa2c0069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59710/0 (socket says 172.21.15.79:59710) 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.468+0000 7fa2bf7fe700 1 -- 172.21.15.79:0/2129202912 learned_addr learned my addr 172.21.15.79:0/2129202912 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2be7fc700 1 --2- 172.21.15.79:0/2129202912 >> v2:172.21.15.156:3300/0 conn(0x7fa2c0071fe0 0x7fa2c00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2beffd700 1 -- 172.21.15.79:0/2129202912 >> v2:172.21.15.156:3300/0 conn(0x7fa2c0071fe0 msgr2=0x7fa2c00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:15.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2beffd700 1 --2- 172.21.15.79:0/2129202912 >> v2:172.21.15.156:3300/0 conn(0x7fa2c0071fe0 0x7fa2c00cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2beffd700 1 -- 172.21.15.79:0/2129202912 >> v1:172.21.15.79:6789/0 conn(0x7fa2c00cf130 legacy=0x7fa2c0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2beffd700 1 -- 172.21.15.79:0/2129202912 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c00cd390 con 0x7fa2c015c1f0 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2bdffb700 1 -- 172.21.15.79:0/2129202912 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409486950 0 0) 0x7fa2c00cd140 con 0x7fa2c00cf130 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2beffd700 1 --2- 172.21.15.79:0/2129202912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c006d8a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d0670 tx=0x7fa2b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afca087732e17c7 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2bdffb700 1 -- 172.21.15.79:0/2129202912 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011830 con 0x7fa2c015c1f0 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2bdffb700 1 -- 172.21.15.79:0/2129202912 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4011990 con 0x7fa2c015c1f0 2023-06-08T03:51:15.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2bdffb700 1 -- 172.21.15.79:0/2129202912 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4011d20 con 0x7fa2c015c1f0 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2c5745700 1 -- 172.21.15.79:0/2129202912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 msgr2=0x7fa2c006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.469+0000 7fa2c5745700 1 --2- 172.21.15.79:0/2129202912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c006d8a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d0670 tx=0x7fa2b4009b20 comp rx=0 tx=0).stop 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.470+0000 7fa2c5745700 1 -- 172.21.15.79:0/2129202912 shutdown_connections 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.470+0000 7fa2c5745700 1 --2- 172.21.15.79:0/2129202912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c006d8a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.470+0000 7fa2c5745700 1 --2- 172.21.15.79:0/2129202912 >> v2:172.21.15.156:3300/0 conn(0x7fa2c0071fe0 0x7fa2c00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.470+0000 7fa2c5745700 1 -- 172.21.15.79:0/2129202912 >> 172.21.15.79:0/2129202912 conn(0x7fa2c00b1d30 msgr2=0x7fa2c006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.470+0000 7fa2c5745700 1 -- 172.21.15.79:0/2129202912 shutdown_connections 2023-06-08T03:51:15.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.470+0000 7fa2c5745700 1 -- 172.21.15.79:0/2129202912 wait complete. 2023-06-08T03:51:15.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.471+0000 7fa2c5745700 1 Processor -- start 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.471+0000 7fa2c5745700 1 -- start start 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2c0071fe0 0x7fa2c00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2c00cf130 0x7fa2c00c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c01f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00b5260 con 0x7fa2c015c1f0 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00c9970 con 0x7fa2c00cf130 2023-06-08T03:51:15.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c00c48b0 con 0x7fa2c0071fe0 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c01f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c01f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36240/0 (socket says 172.21.15.79:36240) 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 -- 172.21.15.79:0/3879925390 learned_addr learned my addr 172.21.15.79:0/3879925390 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2beffd700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2c0071fe0 0x7fa2c00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2be7fc700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2c00cf130 0x7fa2c00c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 -- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2c0071fe0 msgr2=0x7fa2c00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2c0071fe0 0x7fa2c00c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 -- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2c00cf130 msgr2=0x7fa2c00c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2c00cf130 0x7fa2c00c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 -- 172.21.15.79:0/3879925390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40094a0 con 0x7fa2c015c1f0 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2beffd700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2c0071fe0 0x7fa2c00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2bf7fe700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c01f4f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800cb40 tx=0x7fa2b800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800ab50 con 0x7fa2c015c1f0 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b800acd0 con 0x7fa2c015c1f0 2023-06-08T03:51:15.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c00c4b90 con 0x7fa2c015c1f0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8007310 con 0x7fa2c015c1f0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.472+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c00c5150 con 0x7fa2c015c1f0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.473+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c00c5370 con 0x7fa2c015c1f0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.474+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2b8007470 con 0x7fa2c015c1f0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.474+0000 7fa2a77fe700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa2ac06ace0 0x7fa2ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.474+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa2b8089600 con 0x7fa2c015c1f0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.474+0000 7fa2beffd700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa2ac06ace0 0x7fa2ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.475+0000 7fa2beffd700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa2ac06ace0 0x7fa2ac06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40090b0 tx=0x7fa2b401c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.478+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa2b80543c0 con 0x7fa2c015c1f0 2023-06-08T03:51:15.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.664+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa2c0073e90 con 0x7fa2c015c1f0 2023-06-08T03:51:15.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.665+0000 7fa2a77fe700 1 -- 172.21.15.79:0/3879925390 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2b80579e0 con 0x7fa2c015c1f0 2023-06-08T03:51:15.669 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:15.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa2ac06ace0 msgr2=0x7fa2ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa2ac06ace0 0x7fa2ac06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40090b0 tx=0x7fa2b401c040 comp rx=0 tx=0).stop 2023-06-08T03:51:15.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 msgr2=0x7fa2c01f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c01f4f50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800cb40 tx=0x7fa2b800cf00 comp rx=0 tx=0).stop 2023-06-08T03:51:15.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 shutdown_connections 2023-06-08T03:51:15.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c015c1f0 0x7fa2c01f4f50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa2ac06ace0 0x7fa2ac06d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa2c0071fe0 0x7fa2c00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 --2- 172.21.15.79:0/3879925390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2c00cf130 0x7fa2c00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 >> 172.21.15.79:0/3879925390 conn(0x7fa2c00b1d30 msgr2=0x7fa2c015cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:15.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 shutdown_connections 2023-06-08T03:51:15.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.667+0000 7fa2c5745700 1 -- 172.21.15.79:0/3879925390 wait complete. 2023-06-08T03:51:15.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:15.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 Processor -- start 2023-06-08T03:51:15.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 -- start start 2023-06-08T03:51:15.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c90069d50 0x7f0c9006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c900cbbf0 0x7f0c900cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9015c690 con 0x7f0c9006a120 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c900cef60 con 0x7f0c90069d50 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.800+0000 7f0c95e7e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900cf240 con 0x7f0c900cbbf0 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c90069d50 0x7f0c9006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c87fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0c9006a120 0x7f0c90071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59724/0 (socket says 172.21.15.79:59724) 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c87fff700 1 -- 172.21.15.79:0/3495283008 learned_addr learned my addr 172.21.15.79:0/3495283008 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8ffff700 1 --2- 172.21.15.79:0/3495283008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c900cbbf0 0x7f0c900cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8f7fe700 1 -- 172.21.15.79:0/3495283008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c900cbbf0 msgr2=0x7f0c900cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:15.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8f7fe700 1 --2- 172.21.15.79:0/3495283008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c900cbbf0 0x7f0c900cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8f7fe700 1 -- 172.21.15.79:0/3495283008 >> v1:172.21.15.79:6789/0 conn(0x7f0c9006a120 legacy=0x7f0c90071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8f7fe700 1 -- 172.21.15.79:0/3495283008 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9006eac0 con 0x7f0c90069d50 2023-06-08T03:51:15.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8effd700 1 -- 172.21.15.79:0/3495283008 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012272565 0 0) 0x7f0c9015c690 con 0x7f0c9006a120 2023-06-08T03:51:15.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8f7fe700 1 --2- 172.21.15.79:0/3495283008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c90069d50 0x7f0c9006f5a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009840 tx=0x7f0c80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8092111b68fc123a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8effd700 1 -- 172.21.15.79:0/3495283008 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80011830 con 0x7f0c90069d50 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8effd700 1 -- 172.21.15.79:0/3495283008 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c80011990 con 0x7f0c90069d50 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.801+0000 7f0c8effd700 1 -- 172.21.15.79:0/3495283008 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c80011d20 con 0x7f0c90069d50 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 -- 172.21.15.79:0/3495283008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c90069d50 msgr2=0x7f0c9006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/3495283008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c90069d50 0x7f0c9006f5a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009840 tx=0x7f0c80009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 -- 172.21.15.79:0/3495283008 shutdown_connections 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/3495283008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c900cbbf0 0x7f0c900cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/3495283008 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c90069d50 0x7f0c9006f5a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 -- 172.21.15.79:0/3495283008 >> 172.21.15.79:0/3495283008 conn(0x7f0c900b1d30 msgr2=0x7f0c9006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:15.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 -- 172.21.15.79:0/3495283008 shutdown_connections 2023-06-08T03:51:15.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.802+0000 7f0c95e7e700 1 -- 172.21.15.79:0/3495283008 wait complete. 2023-06-08T03:51:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.803+0000 7f0c95e7e700 1 Processor -- start 2023-06-08T03:51:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.803+0000 7f0c95e7e700 1 -- start start 2023-06-08T03:51:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c95e7e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90069d50 0x7f0c900ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c95e7e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c9006a120 0x7f0c90152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c95e7e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 0x7f0c900c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c95e7e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900c59f0 con 0x7f0c900cbbf0 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c95e7e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900ae6f0 con 0x7f0c90069d50 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c95e7e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c90152f50 con 0x7f0c9006a120 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 0x7f0c900c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 0x7f0c900c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36254/0 (socket says 172.21.15.79:36254) 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 -- 172.21.15.79:0/286533697 learned_addr learned my addr 172.21.15.79:0/286533697 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8f7fe700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90069d50 0x7f0c900ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c87fff700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c9006a120 0x7f0c90152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 -- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c9006a120 msgr2=0x7f0c90152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c9006a120 0x7f0c90152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 -- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90069d50 msgr2=0x7f0c900ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90069d50 0x7f0c900ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 -- 172.21.15.79:0/286533697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800094a0 con 0x7f0c900cbbf0 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c87fff700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c9006a120 0x7f0c90152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.804+0000 7f0c8ffff700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 0x7f0c900c30a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c88007e70 tx=0x7f0c8800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.805+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800ede0 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.805+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88014420 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.805+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c900c5e80 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.805+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880147d0 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.805+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c900c53a0 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.806+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c900c51e0 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.806+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c88014930 con 0x7f0c900cbbf0 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.806+0000 7f0c8cff9700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0c7006ace0 0x7f0c7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.806+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0c88089e00 con 0x7f0c900cbbf0 2023-06-08T03:51:15.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.806+0000 7f0c8f7fe700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0c7006ace0 0x7f0c7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:15.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.807+0000 7f0c8f7fe700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0c7006ace0 0x7f0c7006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c800090b0 tx=0x7f0c8001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:15.816 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:15.810+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c88054c50 con 0x7f0c900cbbf0 2023-06-08T03:51:16.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.012+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c90073e90 con 0x7f0c900cbbf0 2023-06-08T03:51:16.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.012+0000 7f0c8cff9700 1 -- 172.21.15.79:0/286533697 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0c88058270 con 0x7f0c900cbbf0 2023-06-08T03:51:16.018 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0c7006ace0 msgr2=0x7f0c7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0c7006ace0 0x7f0c7006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0c800090b0 tx=0x7f0c8001c040 comp rx=0 tx=0).stop 2023-06-08T03:51:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 msgr2=0x7f0c900c30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 0x7f0c900c30a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c88007e70 tx=0x7f0c8800e3c0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 shutdown_connections 2023-06-08T03:51:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c900cbbf0 0x7f0c900c30a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.019 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c90069d50 0x7f0c900ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f0c7006ace0 0x7f0c7006d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 --2- 172.21.15.79:0/286533697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0c9006a120 0x7f0c90152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 >> 172.21.15.79:0/286533697 conn(0x7f0c900b1d30 msgr2=0x7f0c9006d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 shutdown_connections 2023-06-08T03:51:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.015+0000 7f0c95e7e700 1 -- 172.21.15.79:0/286533697 wait complete. 2023-06-08T03:51:16.020 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:16.030 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:51:16.031 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:16.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:16.206 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.203+0000 7f5a22d88700 1 Processor -- start 2023-06-08T03:51:16.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.203+0000 7f5a22d88700 1 -- start start 2023-06-08T03:51:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a22d88700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 0x7f5a1c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a22d88700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a1c070ef0 0x7f5a1c0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a22d88700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c06c7e0 con 0x7f5a1c0cad00 2023-06-08T03:51:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a22d88700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c158a80 con 0x7f5a1c070ef0 2023-06-08T03:51:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a22d88700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0cf910 con 0x7f5a1c1586b0 2023-06-08T03:51:16.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1bfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5a1c0cad00 0x7f5a1c0707e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59736/0 (socket says 172.21.15.79:59736) 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1bfff700 1 -- 172.21.15.79:0/3164342060 learned_addr learned my addr 172.21.15.79:0/3164342060 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428263977 0 0) 0x7f5a1c06c7e0 con 0x7f5a1c0cad00 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a21325700 1 --2- 172.21.15.79:0/3164342060 >> v2:172.21.15.156:3300/0 conn(0x7f5a1c070ef0 0x7f5a1c0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c0035b0 con 0x7f5a1c0cad00 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a20b24700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 0x7f5a1c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148747831 0 0) 0x7f5a0c0035b0 con 0x7f5a1c0cad00 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 msgr2=0x7f5a1c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:16.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 0x7f5a1c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 >> v2:172.21.15.156:3300/0 conn(0x7f5a1c070ef0 msgr2=0x7f5a1c0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 --2- 172.21.15.79:0/3164342060 >> v2:172.21.15.156:3300/0 conn(0x7f5a1c070ef0 0x7f5a1c0c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c06ca30 con 0x7f5a1c0cad00 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f5a08001670 con 0x7f5a1c0cad00 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 >> v1:172.21.15.79:6789/0 conn(0x7f5a1c0cad00 legacy=0x7f5a1c0707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0c003a70 0x7f5a0c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 0x7f5a0c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0cf910 con 0x7f5a0c003a70 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00bac0 con 0x7f5a0c004850 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.204+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c00bc80 con 0x7f5a1c1586b0 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0c003a70 0x7f5a0c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a1bfff700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a21325700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 0x7f5a0c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 -- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 msgr2=0x7f5a0c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a1c1586b0 0x7f5a0c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 -- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a0c004850 msgr2=0x7f5a0c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a0c004850 0x7f5a0c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 -- 172.21.15.79:0/3164342060 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c0035b0 con 0x7f5a0c003a70 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a20b24700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0c003a70 0x7f5a0c003e80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000bd40 tx=0x7f5a0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87acfbdbbce476a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10010d50 con 0x7f5a0c003a70 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1000be90 con 0x7f5a0c003a70 2023-06-08T03:51:16.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a1b7fe700 1 -- 172.21.15.79:0/3164342060 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10019610 con 0x7f5a0c003a70 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a22d88700 1 -- 172.21.15.79:0/3164342060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0c003a70 msgr2=0x7f5a0c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a22d88700 1 --2- 172.21.15.79:0/3164342060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0c003a70 0x7f5a0c003e80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000bd40 tx=0x7f5a0c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.205+0000 7f5a21325700 1 -- 172.21.15.79:0/3164342060 reap_dead start 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.206+0000 7f5a22d88700 1 -- 172.21.15.79:0/3164342060 shutdown_connections 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.206+0000 7f5a22d88700 1 -- 172.21.15.79:0/3164342060 >> 172.21.15.79:0/3164342060 conn(0x7f5a1c0b14a0 msgr2=0x7f5a1c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.206+0000 7f5a22d88700 1 -- 172.21.15.79:0/3164342060 shutdown_connections 2023-06-08T03:51:16.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.206+0000 7f5a22d88700 1 -- 172.21.15.79:0/3164342060 wait complete. 2023-06-08T03:51:16.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.207+0000 7f5a22d88700 1 Processor -- start 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.207+0000 7f5a22d88700 1 -- start start 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0c003a70 0x7f5a1c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c0c6790 0x7f5a1c0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 0x7f5a1c1ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0c75d0 con 0x7f5a1c1ee280 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c1eddf0 con 0x7f5a1c0c6790 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0c6d30 con 0x7f5a0c003a70 2023-06-08T03:51:16.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 0x7f5a1c1ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 0x7f5a1c1ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36274/0 (socket says 172.21.15.79:36274) 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 -- 172.21.15.79:0/1726495161 learned_addr learned my addr 172.21.15.79:0/1726495161 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a1bfff700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c0c6790 0x7f5a1c0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a20b24700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0c003a70 0x7f5a1c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 -- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0c003a70 msgr2=0x7f5a1c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0c003a70 0x7f5a1c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 -- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c0c6790 msgr2=0x7f5a1c0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c0c6790 0x7f5a1c0c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 -- 172.21.15.79:0/1726495161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0c7010 con 0x7f5a1c1ee280 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a20b24700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0c003a70 0x7f5a1c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a21325700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 0x7f5a1c1ee700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c070560 tx=0x7f5a14008cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14019070 con 0x7f5a1c1ee280 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a14010040 con 0x7f5a1c1ee280 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c0c72f0 con 0x7f5a1c1ee280 2023-06-08T03:51:16.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400f050 con 0x7f5a1c1ee280 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.208+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c15a370 con 0x7f5a1c1ee280 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.209+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c1edb90 con 0x7f5a1c1ee280 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.210+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a14003c60 con 0x7f5a1c1ee280 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.210+0000 7f5a197fa700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.210+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5a14089370 con 0x7f5a1c1ee280 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.210+0000 7f5a20b24700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.211+0000 7f5a20b24700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a0006ace0 0x7f5a0006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5a1001b040 tx=0x7f5a10015000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.214+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a140541c0 con 0x7f5a1c1ee280 2023-06-08T03:51:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.401+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a1c06cfc0 con 0x7f5a1c1ee280 2023-06-08T03:51:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.402+0000 7f5a197fa700 1 -- 172.21.15.79:0/1726495161 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f5a1401c310 con 0x7f5a1c1ee280 2023-06-08T03:51:16.406 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:16.407 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:16.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a0006ace0 msgr2=0x7f5a0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a0006ace0 0x7f5a0006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5a1001b040 tx=0x7f5a10015000 comp rx=0 tx=0).stop 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 msgr2=0x7f5a1c1ee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 0x7f5a1c1ee700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c070560 tx=0x7f5a14008cc0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 shutdown_connections 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a1c1ee280 0x7f5a1c1ee700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0c003a70 0x7f5a1c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 --2- 172.21.15.79:0/1726495161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a1c0c6790 0x7f5a1c0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 >> 172.21.15.79:0/1726495161 conn(0x7f5a1c0b14a0 msgr2=0x7f5a1c0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:16.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 shutdown_connections 2023-06-08T03:51:16.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.404+0000 7f5a22d88700 1 -- 172.21.15.79:0/1726495161 wait complete. 2023-06-08T03:51:16.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:16.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:16.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.539+0000 7fd971993700 1 Processor -- start 2023-06-08T03:51:16.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.539+0000 7fd971993700 1 -- start start 2023-06-08T03:51:16.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd971993700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd96c158d50 0x7fd96c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd971993700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c0cd7b0 con 0x7fd96c15d3d0 2023-06-08T03:51:16.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd971993700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c0cda50 con 0x7fd96c0cc690 2023-06-08T03:51:16.545 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd971993700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c0cdd30 con 0x7fd96c158d50 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd96a7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd96c15d3d0 0x7fd96c0cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59756/0 (socket says 172.21.15.79:59756) 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd96a7fc700 1 -- 172.21.15.79:0/2110009303 learned_addr learned my addr 172.21.15.79:0/2110009303 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.540+0000 7fd96affd700 1 --2- 172.21.15.79:0/2110009303 >> v2:172.21.15.156:3300/0 conn(0x7fd96c158d50 0x7fd96c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd969ffb700 1 -- 172.21.15.79:0/2110009303 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692839845 0 0) 0x7fd96c0cd7b0 con 0x7fd96c15d3d0 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd969ffb700 1 -- 172.21.15.79:0/2110009303 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9540035b0 con 0x7fd96c15d3d0 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd969ffb700 1 -- 172.21.15.79:0/2110009303 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001978313 0 0) 0x7fd9540035b0 con 0x7fd96c15d3d0 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd96affd700 1 -- 172.21.15.79:0/2110009303 >> v1:172.21.15.156:6789/0 conn(0x7fd96c0cc690 legacy=0x7fd96c071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd96affd700 1 -- 172.21.15.79:0/2110009303 >> v1:172.21.15.79:6789/0 conn(0x7fd96c15d3d0 legacy=0x7fd96c0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd96affd700 1 -- 172.21.15.79:0/2110009303 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0ce010 con 0x7fd96c158d50 2023-06-08T03:51:16.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd96affd700 1 --2- 172.21.15.79:0/2110009303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c0ca360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd960009840 tx=0x7fd960009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff9e883403a3b9e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd969ffb700 1 -- 172.21.15.79:0/2110009303 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c001670 con 0x7fd96c158d50 2023-06-08T03:51:16.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd969ffb700 1 -- 172.21.15.79:0/2110009303 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd960004030 con 0x7fd96c158d50 2023-06-08T03:51:16.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.541+0000 7fd969ffb700 1 -- 172.21.15.79:0/2110009303 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9600043c0 con 0x7fd96c158d50 2023-06-08T03:51:16.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 -- 172.21.15.79:0/2110009303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 msgr2=0x7fd96c0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 --2- 172.21.15.79:0/2110009303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c0ca360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd960009840 tx=0x7fd960009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 -- 172.21.15.79:0/2110009303 shutdown_connections 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 --2- 172.21.15.79:0/2110009303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c0ca360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 -- 172.21.15.79:0/2110009303 >> 172.21.15.79:0/2110009303 conn(0x7fd96c0b1d30 msgr2=0x7fd96c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 -- 172.21.15.79:0/2110009303 shutdown_connections 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.542+0000 7fd971993700 1 -- 172.21.15.79:0/2110009303 wait complete. 2023-06-08T03:51:16.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.543+0000 7fd971993700 1 Processor -- start 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.543+0000 7fd971993700 1 -- start start 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd971993700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 0x7fd96c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd971993700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd971993700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd96c15d3d0 0x7fd96c1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd971993700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c154c40 con 0x7fd96c0cc690 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd971993700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c159120 con 0x7fd96c158d50 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd971993700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c154e50 con 0x7fd96c15d3d0 2023-06-08T03:51:16.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 0x7fd96c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 0x7fd96c15bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36284/0 (socket says 172.21.15.79:36284) 2023-06-08T03:51:16.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 -- 172.21.15.79:0/1749222888 learned_addr learned my addr 172.21.15.79:0/1749222888 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:16.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96a7fc700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96b7fe700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd96c15d3d0 0x7fd96c1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 -- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd96c15d3d0 msgr2=0x7fd96c1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd96c15d3d0 0x7fd96c1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 -- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 msgr2=0x7fd96c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 -- 172.21.15.79:0/1749222888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600094a0 con 0x7fd96c0cc690 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96a7fc700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96b7fe700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd96c15d3d0 0x7fd96c1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd96affd700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 0x7fd96c15bbe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd960009b90 tx=0x7fd960004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.544+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960011830 con 0x7fd96c0cc690 2023-06-08T03:51:16.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.545+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96001f070 con 0x7fd96c0cc690 2023-06-08T03:51:16.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.545+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c155130 con 0x7fd96c0cc690 2023-06-08T03:51:16.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.545+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960011bd0 con 0x7fd96c0cc690 2023-06-08T03:51:16.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.545+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c156010 con 0x7fd96c0cc690 2023-06-08T03:51:16.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.546+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c15d850 con 0x7fd96c0cc690 2023-06-08T03:51:16.554 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.546+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd960011d30 con 0x7fd96c0cc690 2023-06-08T03:51:16.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.546+0000 7fd953fff700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd95806ace0 0x7fd95806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.546+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd960021070 con 0x7fd96c0cc690 2023-06-08T03:51:16.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.546+0000 7fd96a7fc700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd95806ace0 0x7fd95806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.547+0000 7fd96a7fc700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd95806ace0 0x7fd95806d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd96c1564d0 tx=0x7fd95c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.551+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd9600bd310 con 0x7fd96c0cc690 2023-06-08T03:51:16.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.737+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd96c1561a0 con 0x7fd96c0cc690 2023-06-08T03:51:16.741 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.737+0000 7fd953fff700 1 -- 172.21.15.79:0/1749222888 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd96c15d850 con 0x7fd96c0cc690 2023-06-08T03:51:16.741 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:16.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd95806ace0 msgr2=0x7fd95806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd95806ace0 0x7fd95806d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd96c1564d0 tx=0x7fd95c009040 comp rx=0 tx=0).stop 2023-06-08T03:51:16.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 msgr2=0x7fd96c15bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 0x7fd96c15bbe0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd960009b90 tx=0x7fd960004750 comp rx=0 tx=0).stop 2023-06-08T03:51:16.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 shutdown_connections 2023-06-08T03:51:16.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd95806ace0 0x7fd95806d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd96c15d3d0 0x7fd96c1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.744 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd96c0cc690 0x7fd96c15bbe0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 --2- 172.21.15.79:0/1749222888 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd96c158d50 0x7fd96c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 >> 172.21.15.79:0/1749222888 conn(0x7fd96c0b1d30 msgr2=0x7fd96c156670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:16.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 shutdown_connections 2023-06-08T03:51:16.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.739+0000 7fd971993700 1 -- 172.21.15.79:0/1749222888 wait complete. 2023-06-08T03:51:16.755 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:16.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.874+0000 7fe5f0497700 1 Processor -- start 2023-06-08T03:51:16.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.874+0000 7fe5f0497700 1 -- start start 2023-06-08T03:51:16.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5f0497700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe5e80d0240 0x7fe5e80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5f0497700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80cd000 0x7fe5e80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5f0497700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e815d930 con 0x7fe5e80b20f0 2023-06-08T03:51:16.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5f0497700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e815e770 con 0x7fe5e80d0240 2023-06-08T03:51:16.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5f0497700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80cd9b0 con 0x7fe5e80cd000 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5eea34700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe5e80b20f0 0x7fe5e80b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59770/0 (socket says 172.21.15.79:59770) 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5eea34700 1 -- 172.21.15.79:0/418203896 learned_addr learned my addr 172.21.15.79:0/418203896 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5ee233700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe5e80d0240 0x7fe5e80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.875+0000 7fe5eda32700 1 --2- 172.21.15.79:0/418203896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80cd000 0x7fe5e80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ed231700 1 -- 172.21.15.79:0/418203896 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737844637 0 0) 0x7fe5e815d930 con 0x7fe5e80b20f0 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ed231700 1 -- 172.21.15.79:0/418203896 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d40035b0 con 0x7fe5e80b20f0 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ee233700 1 -- 172.21.15.79:0/418203896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80cd000 msgr2=0x7fe5e80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:16.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ee233700 1 --2- 172.21.15.79:0/418203896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80cd000 0x7fe5e80cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ee233700 1 -- 172.21.15.79:0/418203896 >> v1:172.21.15.79:6789/0 conn(0x7fe5e80b20f0 legacy=0x7fe5e80b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ee233700 1 -- 172.21.15.79:0/418203896 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e815db60 con 0x7fe5e80d0240 2023-06-08T03:51:16.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ee233700 1 --2- 172.21.15.79:0/418203896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80d0240 0x7fe5e80af7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009840 tx=0x7fe5d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1988454b9cda47d0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ed231700 1 -- 172.21.15.79:0/418203896 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8011830 con 0x7fe5e80d0240 2023-06-08T03:51:16.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ed231700 1 -- 172.21.15.79:0/418203896 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d8011990 con 0x7fe5e80d0240 2023-06-08T03:51:16.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5ed231700 1 -- 172.21.15.79:0/418203896 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8011d20 con 0x7fe5e80d0240 2023-06-08T03:51:16.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5f0497700 1 -- 172.21.15.79:0/418203896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80d0240 msgr2=0x7fe5e80af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.876+0000 7fe5f0497700 1 --2- 172.21.15.79:0/418203896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80d0240 0x7fe5e80af7d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009840 tx=0x7fe5d8009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:16.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.877+0000 7fe5f0497700 1 -- 172.21.15.79:0/418203896 shutdown_connections 2023-06-08T03:51:16.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.877+0000 7fe5f0497700 1 --2- 172.21.15.79:0/418203896 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80d0240 0x7fe5e80af7d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.877+0000 7fe5f0497700 1 --2- 172.21.15.79:0/418203896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80cd000 0x7fe5e80cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.877+0000 7fe5f0497700 1 -- 172.21.15.79:0/418203896 >> 172.21.15.79:0/418203896 conn(0x7fe5e806e1e0 msgr2=0x7fe5e806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.877+0000 7fe5f0497700 1 -- 172.21.15.79:0/418203896 shutdown_connections 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.877+0000 7fe5f0497700 1 -- 172.21.15.79:0/418203896 wait complete. 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.878+0000 7fe5f0497700 1 Processor -- start 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.878+0000 7fe5f0497700 1 -- start start 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.878+0000 7fe5f0497700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 0x7fe5e806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.878+0000 7fe5f0497700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e80cd000 0x7fe5e8152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.878+0000 7fe5f0497700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80d0240 0x7fe5e80c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5f0497700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e8153420 con 0x7fe5e80b20f0 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5f0497700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80c64f0 con 0x7fe5e80cd000 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5f0497700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e80c58e0 con 0x7fe5e80d0240 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 0x7fe5e806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 0x7fe5e806ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36296/0 (socket says 172.21.15.79:36296) 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 -- 172.21.15.79:0/100214980 learned_addr learned my addr 172.21.15.79:0/100214980 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5eda32700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e80cd000 0x7fe5e8152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5eea34700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80d0240 0x7fe5e80c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 -- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80d0240 msgr2=0x7fe5e80c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80d0240 0x7fe5e80c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 -- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e80cd000 msgr2=0x7fe5e8152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e80cd000 0x7fe5e8152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 -- 172.21.15.79:0/100214980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80094a0 con 0x7fe5e80b20f0 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5eda32700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e80cd000 0x7fe5e8152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:16.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5eea34700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80d0240 0x7fe5e80c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5ee233700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 0x7fe5e806ba30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8000c00 tx=0x7fe5d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80036a0 con 0x7fe5e80b20f0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d801f070 con 0x7fe5e80b20f0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e80c5bc0 con 0x7fe5e80b20f0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8003a40 con 0x7fe5e80b20f0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.879+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e80c6180 con 0x7fe5e80b20f0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.880+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e80c6310 con 0x7fe5e80b20f0 2023-06-08T03:51:16.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.880+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5d8003ba0 con 0x7fe5e80b20f0 2023-06-08T03:51:16.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.881+0000 7fe5deffd700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5cc06ace0 0x7fe5cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:16.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.881+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe5d8021070 con 0x7fe5e80b20f0 2023-06-08T03:51:16.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.881+0000 7fe5eda32700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5cc06ace0 0x7fe5cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:16.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.881+0000 7fe5eda32700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5cc06ace0 0x7fe5cc06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5e806c4e0 tx=0x7fe5e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:16.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:16.885+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5d8055e40 con 0x7fe5e80b20f0 2023-06-08T03:51:17.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.076+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5e8073e90 con 0x7fe5e80b20f0 2023-06-08T03:51:17.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.077+0000 7fe5deffd700 1 -- 172.21.15.79:0/100214980 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5d80150a0 con 0x7fe5e80b20f0 2023-06-08T03:51:17.081 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:17.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5cc06ace0 msgr2=0x7fe5cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5cc06ace0 0x7fe5cc06d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe5e806c4e0 tx=0x7fe5e4008040 comp rx=0 tx=0).stop 2023-06-08T03:51:17.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 msgr2=0x7fe5e806ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 0x7fe5e806ba30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8000c00 tx=0x7fe5d8004750 comp rx=0 tx=0).stop 2023-06-08T03:51:17.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 shutdown_connections 2023-06-08T03:51:17.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe5cc06ace0 0x7fe5cc06d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5e80d0240 0x7fe5e80c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe5e80b20f0 0x7fe5e806ba30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 --2- 172.21.15.79:0/100214980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e80cd000 0x7fe5e8152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 >> 172.21.15.79:0/100214980 conn(0x7fe5e806e1e0 msgr2=0x7fe5e8159890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 shutdown_connections 2023-06-08T03:51:17.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.079+0000 7fe5f0497700 1 -- 172.21.15.79:0/100214980 wait complete. 2023-06-08T03:51:17.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:17.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.214+0000 7f8f3727e700 1 Processor -- start 2023-06-08T03:51:17.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.214+0000 7f8f3727e700 1 -- start start 2023-06-08T03:51:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.214+0000 7f8f3727e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8f3015c1f0 0x7f8f3006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.214+0000 7f8f3727e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8f30071fe0 0x7f8f300cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3727e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f30069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3727e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f300cd140 con 0x7f8f30071fe0 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3727e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f30071110 con 0x7f8f3015c1f0 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3727e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f30071be0 con 0x7f8f300cf130 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8f30071fe0 0x7f8f300cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8f30071fe0 0x7f8f300cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36304/0 (socket says 172.21.15.79:36304) 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 -- 172.21.15.79:0/2236701775 learned_addr learned my addr 172.21.15.79:0/2236701775 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:17.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3501a700 1 --2- 172.21.15.79:0/2236701775 >> v2:172.21.15.156:3300/0 conn(0x7f8f3015c1f0 0x7f8f3006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3581b700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f30069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 -- 172.21.15.79:0/2236701775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 msgr2=0x7f8f30069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f30069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 -- 172.21.15.79:0/2236701775 >> v2:172.21.15.156:3300/0 conn(0x7f8f3015c1f0 msgr2=0x7f8f3006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 --2- 172.21.15.79:0/2236701775 >> v2:172.21.15.156:3300/0 conn(0x7f8f3015c1f0 0x7f8f3006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 -- 172.21.15.79:0/2236701775 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300cd390 con 0x7f8f30071fe0 2023-06-08T03:51:17.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3581b700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f30069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f3501a700 1 --2- 172.21.15.79:0/2236701775 >> v2:172.21.15.156:3300/0 conn(0x7f8f3015c1f0 0x7f8f3006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f34819700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f300cebf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f30070d60 tx=0x7f8f2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffebe2a1af52772 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f27fff700 1 -- 172.21.15.79:0/2236701775 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00b870 con 0x7f8f30071fe0 2023-06-08T03:51:17.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f27fff700 1 -- 172.21.15.79:0/2236701775 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c00b9d0 con 0x7f8f30071fe0 2023-06-08T03:51:17.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.215+0000 7f8f27fff700 1 -- 172.21.15.79:0/2236701775 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00bd60 con 0x7f8f30071fe0 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 -- 172.21.15.79:0/2236701775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 msgr2=0x7f8f300cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f300cebf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f30070d60 tx=0x7f8f2c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 -- 172.21.15.79:0/2236701775 shutdown_connections 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f30069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 --2- 172.21.15.79:0/2236701775 >> v2:172.21.15.156:3300/0 conn(0x7f8f3015c1f0 0x7f8f3006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 --2- 172.21.15.79:0/2236701775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f300cebf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 -- 172.21.15.79:0/2236701775 >> 172.21.15.79:0/2236701775 conn(0x7f8f300b1d30 msgr2=0x7f8f3006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 -- 172.21.15.79:0/2236701775 shutdown_connections 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.216+0000 7f8f3727e700 1 -- 172.21.15.79:0/2236701775 wait complete. 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.217+0000 7f8f3727e700 1 Processor -- start 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.217+0000 7f8f3727e700 1 -- start start 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f30151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f300b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3015c1f0 0x7f8f300b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301517d0 con 0x7f8f30071fe0 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300c7570 con 0x7f8f3015c1f0 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300c6f90 con 0x7f8f300cf130 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f30151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f30151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36308/0 (socket says 172.21.15.79:36308) 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 -- 172.21.15.79:0/137096345 learned_addr learned my addr 172.21.15.79:0/137096345 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3581b700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3015c1f0 0x7f8f300b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f34819700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f300b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 -- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 msgr2=0x7f8f300b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f300b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 -- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3015c1f0 msgr2=0x7f8f300b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3015c1f0 0x7f8f300b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 -- 172.21.15.79:0/137096345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c009550 con 0x7f8f30071fe0 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f34819700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f300b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3581b700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3015c1f0 0x7f8f300b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3501a700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f30151290 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000c780 tx=0x7f8f2000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20011800 con 0x7f8f30071fe0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f20011960 con 0x7f8f30071fe0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f300b2d10 con 0x7f8f30071fe0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.218+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20011cd0 con 0x7f8f30071fe0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.219+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f301f1ff0 con 0x7f8f30071fe0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.220+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f30073e90 con 0x7f8f30071fe0 2023-06-08T03:51:17.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.220+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f20021350 con 0x7f8f30071fe0 2023-06-08T03:51:17.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.220+0000 7f8f25ffb700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1406ace0 0x7f8f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.220+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8f20089700 con 0x7f8f30071fe0 2023-06-08T03:51:17.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.221+0000 7f8f34819700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1406ace0 0x7f8f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.221+0000 7f8f34819700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1406ace0 0x7f8f1406d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f3006da20 tx=0x7f8f2c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.230 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.225+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f20057f60 con 0x7f8f30071fe0 2023-06-08T03:51:17.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.409+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f300c96e0 con 0x7f8f30071fe0 2023-06-08T03:51:17.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.410+0000 7f8f25ffb700 1 -- 172.21.15.79:0/137096345 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f20057af0 con 0x7f8f30071fe0 2023-06-08T03:51:17.414 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:17.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1406ace0 msgr2=0x7f8f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1406ace0 0x7f8f1406d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f3006da20 tx=0x7f8f2c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:17.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 msgr2=0x7f8f30151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f30151290 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8f2000c780 tx=0x7f8f2000cb40 comp rx=0 tx=0).stop 2023-06-08T03:51:17.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 shutdown_connections 2023-06-08T03:51:17.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3015c1f0 0x7f8f300b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f30071fe0 0x7f8f30151290 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1406ace0 0x7f8f1406d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 --2- 172.21.15.79:0/137096345 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f300cf130 0x7f8f300b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 >> 172.21.15.79:0/137096345 conn(0x7f8f300b1d30 msgr2=0x7f8f300cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:17.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 shutdown_connections 2023-06-08T03:51:17.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.412+0000 7f8f3727e700 1 -- 172.21.15.79:0/137096345 wait complete. 2023-06-08T03:51:17.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:17.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.545+0000 7fbf8affb700 1 Processor -- start 2023-06-08T03:51:17.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.545+0000 7fbf8affb700 1 -- start start 2023-06-08T03:51:17.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf8affb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbf8415c1f0 0x7fbf8406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf8affb700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf840cd140 con 0x7fbf84069d50 2023-06-08T03:51:17.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf8affb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf84071110 con 0x7fbf84071fe0 2023-06-08T03:51:17.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf8affb700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf84071be0 con 0x7fbf8415c1f0 2023-06-08T03:51:17.551 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf88d97700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbf8415c1f0 0x7fbf8406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf88d97700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbf8415c1f0 0x7fbf8406d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36316/0 (socket says 172.21.15.79:36316) 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf88d97700 1 -- 172.21.15.79:0/3833673865 learned_addr learned my addr 172.21.15.79:0/3833673865 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.546+0000 7fbf89598700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbf84069d50 0x7fbf840cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59792/0 (socket says 172.21.15.79:59792) 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf88d97700 1 -- 172.21.15.79:0/3833673865 >> v1:172.21.15.156:6789/0 conn(0x7fbf84071fe0 legacy=0x7fbf840cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf88d97700 1 -- 172.21.15.79:0/3833673865 >> v1:172.21.15.79:6789/0 conn(0x7fbf84069d50 legacy=0x7fbf840cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf88d97700 1 -- 172.21.15.79:0/3833673865 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840cd390 con 0x7fbf8415c1f0 2023-06-08T03:51:17.552 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf88d97700 1 --2- 172.21.15.79:0/3833673865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf8415c1f0 0x7fbf8406d8a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf780098f0 tx=0x7fbf78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e30909b540a228a server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf837fe700 1 -- 172.21.15.79:0/3833673865 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78004230 con 0x7fbf8415c1f0 2023-06-08T03:51:17.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf837fe700 1 -- 172.21.15.79:0/3833673865 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf78004390 con 0x7fbf8415c1f0 2023-06-08T03:51:17.553 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf837fe700 1 -- 172.21.15.79:0/3833673865 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780036a0 con 0x7fbf8415c1f0 2023-06-08T03:51:17.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 -- 172.21.15.79:0/3833673865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf8415c1f0 msgr2=0x7fbf8406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.555 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 --2- 172.21.15.79:0/3833673865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf8415c1f0 0x7fbf8406d8a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbf780098f0 tx=0x7fbf78009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 -- 172.21.15.79:0/3833673865 shutdown_connections 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 --2- 172.21.15.79:0/3833673865 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf8415c1f0 0x7fbf8406d8a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 -- 172.21.15.79:0/3833673865 >> 172.21.15.79:0/3833673865 conn(0x7fbf840b1d30 msgr2=0x7fbf8406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 -- 172.21.15.79:0/3833673865 shutdown_connections 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.547+0000 7fbf8affb700 1 -- 172.21.15.79:0/3833673865 wait complete. 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.549+0000 7fbf8affb700 1 Processor -- start 2023-06-08T03:51:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.549+0000 7fbf8affb700 1 -- start start 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.549+0000 7fbf8affb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf84069d50 0x7fbf841519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.549+0000 7fbf8affb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 0x7fbf840c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.549+0000 7fbf8affb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf8415c1f0 0x7fbf840c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.549+0000 7fbf8affb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840c5da0 con 0x7fbf84071fe0 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf8affb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840c8160 con 0x7fbf84069d50 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf8affb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf840c4110 con 0x7fbf8415c1f0 2023-06-08T03:51:17.557 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 0x7fbf840c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 0x7fbf840c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36322/0 (socket says 172.21.15.79:36322) 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 -- 172.21.15.79:0/1904258469 learned_addr learned my addr 172.21.15.79:0/1904258469 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf89598700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf8415c1f0 0x7fbf840c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf88d97700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf84069d50 0x7fbf841519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 -- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf8415c1f0 msgr2=0x7fbf840c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf8415c1f0 0x7fbf840c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.558 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 -- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf84069d50 msgr2=0x7fbf841519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf84069d50 0x7fbf841519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 -- 172.21.15.79:0/1904258469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78009550 con 0x7fbf84071fe0 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf89598700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf8415c1f0 0x7fbf840c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf88d97700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf84069d50 0x7fbf841519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf83fff700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 0x7fbf840c3bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000b4a0 tx=0x7fbf7000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70011830 con 0x7fbf84071fe0 2023-06-08T03:51:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf70011990 con 0x7fbf84071fe0 2023-06-08T03:51:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf840c7830 con 0x7fbf84071fe0 2023-06-08T03:51:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70011d40 con 0x7fbf84071fe0 2023-06-08T03:51:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.550+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf840c7df0 con 0x7fbf84071fe0 2023-06-08T03:51:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.551+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf84073e90 con 0x7fbf84071fe0 2023-06-08T03:51:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.552+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf7001a400 con 0x7fbf84071fe0 2023-06-08T03:51:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.552+0000 7fbf817fa700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbf6806ace0 0x7fbf6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.552+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbf70089500 con 0x7fbf84071fe0 2023-06-08T03:51:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.552+0000 7fbf88d97700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbf6806ace0 0x7fbf6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.553+0000 7fbf88d97700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbf6806ace0 0x7fbf6806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009c40 tx=0x7fbf78008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.556+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf700542c0 con 0x7fbf84071fe0 2023-06-08T03:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.744+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf8406ed70 con 0x7fbf84071fe0 2023-06-08T03:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.744+0000 7fbf817fa700 1 -- 172.21.15.79:0/1904258469 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf700578e0 con 0x7fbf84071fe0 2023-06-08T03:51:17.749 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:17.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbf6806ace0 msgr2=0x7fbf6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbf6806ace0 0x7fbf6806d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009c40 tx=0x7fbf78008030 comp rx=0 tx=0).stop 2023-06-08T03:51:17.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 msgr2=0x7fbf840c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 0x7fbf840c3bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf7000b4a0 tx=0x7fbf7000b860 comp rx=0 tx=0).stop 2023-06-08T03:51:17.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 shutdown_connections 2023-06-08T03:51:17.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbf6806ace0 0x7fbf6806d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf8415c1f0 0x7fbf840c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf84069d50 0x7fbf841519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 --2- 172.21.15.79:0/1904258469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbf84071fe0 0x7fbf840c3bd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 >> 172.21.15.79:0/1904258469 conn(0x7fbf840b1d30 msgr2=0x7fbf840b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:17.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 shutdown_connections 2023-06-08T03:51:17.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.747+0000 7fbf8affb700 1 -- 172.21.15.79:0/1904258469 wait complete. 2023-06-08T03:51:17.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:17.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.881+0000 7f1f2e5f7700 1 Processor -- start 2023-06-08T03:51:17.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.881+0000 7f1f2e5f7700 1 -- start start 2023-06-08T03:51:17.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f2e5f7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1f2815c0a0 0x7f1f28070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f2e5f7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f2e5f7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f280cd820 con 0x7f1f280cd3c0 2023-06-08T03:51:17.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f2e5f7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2815c470 con 0x7f1f2815c0a0 2023-06-08T03:51:17.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f2e5f7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280cacc0 con 0x7f1f2815fb10 2023-06-08T03:51:17.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f27fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1f2815c0a0 0x7f1f28070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f27fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1f2815c0a0 0x7f1f28070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36328/0 (socket says 172.21.15.79:36328) 2023-06-08T03:51:17.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f27fff700 1 -- 172.21.15.79:0/986689427 learned_addr learned my addr 172.21.15.79:0/986689427 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:17.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.882+0000 7f1f277fe700 1 --2- 172.21.15.79:0/986689427 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f27fff700 1 -- 172.21.15.79:0/986689427 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 msgr2=0x7f1f280cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:17.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f27fff700 1 --2- 172.21.15.79:0/986689427 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f27fff700 1 -- 172.21.15.79:0/986689427 >> v1:172.21.15.156:6789/0 conn(0x7f1f280cd3c0 legacy=0x7f1f280b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f27fff700 1 -- 172.21.15.79:0/986689427 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28070bd0 con 0x7f1f2815c0a0 2023-06-08T03:51:17.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f27fff700 1 --2- 172.21.15.79:0/986689427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f2815c0a0 0x7f1f28070690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0098f0 tx=0x7f1f1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945cd175e20651d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f26ffd700 1 -- 172.21.15.79:0/986689427 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c011830 con 0x7f1f2815c0a0 2023-06-08T03:51:17.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f26ffd700 1 -- 172.21.15.79:0/986689427 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c011990 con 0x7f1f2815c0a0 2023-06-08T03:51:17.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f26ffd700 1 -- 172.21.15.79:0/986689427 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c011d20 con 0x7f1f2815c0a0 2023-06-08T03:51:17.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/986689427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f2815c0a0 msgr2=0x7f1f28070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/986689427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f2815c0a0 0x7f1f28070690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0098f0 tx=0x7f1f1c009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:17.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/986689427 shutdown_connections 2023-06-08T03:51:17.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/986689427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f2815c0a0 0x7f1f28070690 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/986689427 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.883+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/986689427 >> 172.21.15.79:0/986689427 conn(0x7f1f2806e1e0 msgr2=0x7f1f2806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.884+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/986689427 shutdown_connections 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.884+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/986689427 wait complete. 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.885+0000 7f1f2e5f7700 1 Processor -- start 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.885+0000 7f1f2e5f7700 1 -- start start 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 0x7f1f28155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2815c0a0 0x7f1f280c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280c8db0 con 0x7f1f280cd3c0 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280c9450 con 0x7f1f2815c0a0 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280b0ef0 con 0x7f1f2815fb10 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 0x7f1f28155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 0x7f1f28155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36334/0 (socket says 172.21.15.79:36334) 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 -- 172.21.15.79:0/3375626388 learned_addr learned my addr 172.21.15.79:0/3375626388 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:17.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f277fe700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2815c0a0 0x7f1f280c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 -- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 msgr2=0x7f1f280b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2cb94700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 -- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2815c0a0 msgr2=0x7f1f280c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2815c0a0 0x7f1f280c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 -- 172.21.15.79:0/3375626388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c009550 con 0x7f1f280cd3c0 2023-06-08T03:51:17.895 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f277fe700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2815c0a0 0x7f1f280c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2cb94700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f27fff700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 0x7f1f28155d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009c40 tx=0x7f1f1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c004530 con 0x7f1f280cd3c0 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c01f070 con 0x7f1f280cd3c0 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f280ca450 con 0x7f1f280cd3c0 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0038c0 con 0x7f1f280cd3c0 2023-06-08T03:51:17.896 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.886+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f280c8ad0 con 0x7f1f280cd3c0 2023-06-08T03:51:17.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.887+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f28073e90 con 0x7f1f280cd3c0 2023-06-08T03:51:17.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.888+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f1c021030 con 0x7f1f280cd3c0 2023-06-08T03:51:17.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.888+0000 7f1f24ff9700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f1406ace0 0x7f1f1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:17.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.888+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f1c08aac0 con 0x7f1f280cd3c0 2023-06-08T03:51:17.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.888+0000 7f1f277fe700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f1406ace0 0x7f1f1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:17.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.889+0000 7f1f277fe700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f1406ace0 0x7f1f1406d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f280afc20 tx=0x7f1f18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:17.898 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:17.892+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1f1c055990 con 0x7f1f280cd3c0 2023-06-08T03:51:18.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.076+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1f28070bd0 con 0x7f1f280cd3c0 2023-06-08T03:51:18.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.076+0000 7f1f24ff9700 1 -- 172.21.15.79:0/3375626388 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1f1c015020 con 0x7f1f280cd3c0 2023-06-08T03:51:18.080 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:18.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f1406ace0 msgr2=0x7f1f1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f1406ace0 0x7f1f1406d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1f280afc20 tx=0x7f1f18008040 comp rx=0 tx=0).stop 2023-06-08T03:51:18.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 msgr2=0x7f1f28155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 0x7f1f28155d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009c40 tx=0x7f1f1c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 shutdown_connections 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1f1406ace0 0x7f1f1406d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f2815fb10 0x7f1f280b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f280cd3c0 0x7f1f28155d80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 --2- 172.21.15.79:0/3375626388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f2815c0a0 0x7f1f280c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 >> 172.21.15.79:0/3375626388 conn(0x7f1f2806e1e0 msgr2=0x7f1f2806ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:18.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 shutdown_connections 2023-06-08T03:51:18.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.078+0000 7f1f2e5f7700 1 -- 172.21.15.79:0/3375626388 wait complete. 2023-06-08T03:51:18.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:18.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.212+0000 7f0195760700 1 Processor -- start 2023-06-08T03:51:18.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.212+0000 7f0195760700 1 -- start start 2023-06-08T03:51:18.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f0195760700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f019015f820 0x7f01900cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f0195760700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01900d0b80 0x7f01900b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f0195760700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01900b5940 con 0x7f01900ccef0 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f0195760700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01900cef00 con 0x7f019015f820 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f0195760700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900ce660 con 0x7f01900d0b80 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f019015f820 0x7f01900cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018e7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f01900ccef0 0x7f01900d0470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59820/0 (socket says 172.21.15.79:59820) 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018e7fc700 1 -- 172.21.15.79:0/3348953589 learned_addr learned my addr 172.21.15.79:0/3348953589 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018f7fe700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01900d0b80 0x7f01900b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018effd700 1 -- 172.21.15.79:0/3348953589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01900d0b80 msgr2=0x7f01900b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018effd700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01900d0b80 0x7f01900b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018effd700 1 -- 172.21.15.79:0/3348953589 >> v1:172.21.15.79:6789/0 conn(0x7f01900ccef0 legacy=0x7f01900d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018dffb700 1 -- 172.21.15.79:0/3348953589 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624844447 0 0) 0x7f01900b5940 con 0x7f01900ccef0 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018effd700 1 -- 172.21.15.79:0/3348953589 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900ce940 con 0x7f019015f820 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018f7fe700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01900d0b80 0x7f01900b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018effd700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f019015f820 0x7f01900cdde0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0184009820 tx=0x7f0184009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb162aafec4333f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:18.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018dffb700 1 -- 172.21.15.79:0/3348953589 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184005e30 con 0x7f019015f820 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018dffb700 1 -- 172.21.15.79:0/3348953589 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0184006030 con 0x7f019015f820 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.213+0000 7f018dffb700 1 -- 172.21.15.79:0/3348953589 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01840063e0 con 0x7f019015f820 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 -- 172.21.15.79:0/3348953589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f019015f820 msgr2=0x7f01900cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f019015f820 0x7f01900cdde0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0184009820 tx=0x7f0184009b30 comp rx=0 tx=0).stop 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 -- 172.21.15.79:0/3348953589 shutdown_connections 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01900d0b80 0x7f01900b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 --2- 172.21.15.79:0/3348953589 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f019015f820 0x7f01900cdde0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 -- 172.21.15.79:0/3348953589 >> 172.21.15.79:0/3348953589 conn(0x7f019009bc40 msgr2=0x7f01900d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:18.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 -- 172.21.15.79:0/3348953589 shutdown_connections 2023-06-08T03:51:18.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.214+0000 7f0195760700 1 -- 172.21.15.79:0/3348953589 wait complete. 2023-06-08T03:51:18.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.216+0000 7f0195760700 1 Processor -- start 2023-06-08T03:51:18.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.216+0000 7f0195760700 1 -- start start 2023-06-08T03:51:18.224 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01900ccef0 0x7f01900cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 0x7f01900c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f019015f820 0x7f01901497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190159090 con 0x7f01900d0b80 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900c7e30 con 0x7f01900ccef0 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01901587f0 con 0x7f019015f820 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 0x7f01900c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.225 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 0x7f01900c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36350/0 (socket says 172.21.15.79:36350) 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 -- 172.21.15.79:0/608669193 learned_addr learned my addr 172.21.15.79:0/608669193 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018effd700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01900ccef0 0x7f01900cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018f7fe700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f019015f820 0x7f01901497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 -- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f019015f820 msgr2=0x7f01901497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f019015f820 0x7f01901497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 -- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01900ccef0 msgr2=0x7f01900cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.226 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01900ccef0 0x7f01900cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 -- 172.21.15.79:0/608669193 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184009430 con 0x7f01900d0b80 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f018e7fc700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 0x7f01900c8840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f018000c550 tx=0x7f018000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180004c80 con 0x7f01900d0b80 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0180004e00 con 0x7f01900d0b80 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0190158ad0 con 0x7f01900d0b80 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180007310 con 0x7f01900d0b80 2023-06-08T03:51:18.227 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.217+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01900c8040 con 0x7f01900d0b80 2023-06-08T03:51:18.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.218+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0190073e90 con 0x7f01900d0b80 2023-06-08T03:51:18.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.219+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0180007470 con 0x7f01900d0b80 2023-06-08T03:51:18.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.219+0000 7f01777fe700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f017c06ace0 0x7f017c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.219+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0180089420 con 0x7f01900d0b80 2023-06-08T03:51:18.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.219+0000 7f018effd700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f017c06ace0 0x7f017c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.228 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.220+0000 7f018effd700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f017c06ace0 0x7f017c06d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0184013040 tx=0x7f0184008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:18.229 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.224+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f01800541e0 con 0x7f01900d0b80 2023-06-08T03:51:18.431 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.427+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f01900cfb10 con 0x7f01900d0b80 2023-06-08T03:51:18.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.428+0000 7f01777fe700 1 -- 172.21.15.79:0/608669193 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0180057800 con 0x7f01900d0b80 2023-06-08T03:51:18.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f017c06ace0 msgr2=0x7f017c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f017c06ace0 0x7f017c06d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0184013040 tx=0x7f0184008070 comp rx=0 tx=0).stop 2023-06-08T03:51:18.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 msgr2=0x7f01900c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 0x7f01900c8840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f018000c550 tx=0x7f018000c910 comp rx=0 tx=0).stop 2023-06-08T03:51:18.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 shutdown_connections 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f017c06ace0 0x7f017c06d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f019015f820 0x7f01901497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01900ccef0 0x7f01900cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 --2- 172.21.15.79:0/608669193 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01900d0b80 0x7f01900c8840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 >> 172.21.15.79:0/608669193 conn(0x7f019009bc40 msgr2=0x7f01900b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 shutdown_connections 2023-06-08T03:51:18.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.430+0000 7f0195760700 1 -- 172.21.15.79:0/608669193 wait complete. 2023-06-08T03:51:18.436 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:18.445 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2023-06-08T03:51:18.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:18.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:18.485 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:18.485 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:51:18.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:18.486 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-08T03:51:18.541 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:51:18.542 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:18.563 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:51:18.563 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:18.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:18.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.739+0000 7fb39c54a700 1 Processor -- start 2023-06-08T03:51:18.743 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.739+0000 7fb39c54a700 1 -- start start 2023-06-08T03:51:18.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39c54a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb394158e30 0x7fb3940ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39c54a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb394159200 0x7fb3940cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.745 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39c54a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb39415d730 0x7fb3940cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39c54a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3940caef0 con 0x7fb39415d730 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39c54a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3940cb1d0 con 0x7fb394158e30 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39c54a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940cb4b0 con 0x7fb394159200 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb39415d730 0x7fb3940cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb39415d730 0x7fb3940cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36360/0 (socket says 172.21.15.79:36360) 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 -- 172.21.15.79:0/1280583823 learned_addr learned my addr 172.21.15.79:0/1280583823 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 -- 172.21.15.79:0/1280583823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb394159200 msgr2=0x7fb3940cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:18.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb399ae5700 1 --2- 172.21.15.79:0/1280583823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb394159200 0x7fb3940cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39a2e6700 1 --2- 172.21.15.79:0/1280583823 >> v2:172.21.15.156:3300/0 conn(0x7fb394158e30 0x7fb3940ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 --2- 172.21.15.79:0/1280583823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb394159200 0x7fb3940cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 -- 172.21.15.79:0/1280583823 >> v2:172.21.15.156:3300/0 conn(0x7fb394158e30 msgr2=0x7fb3940ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 --2- 172.21.15.79:0/1280583823 >> v2:172.21.15.156:3300/0 conn(0x7fb394158e30 0x7fb3940ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 -- 172.21.15.79:0/1280583823 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3940cb790 con 0x7fb39415d730 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39a2e6700 1 --2- 172.21.15.79:0/1280583823 >> v2:172.21.15.156:3300/0 conn(0x7fb394158e30 0x7fb3940ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:18.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb39aae7700 1 --2- 172.21.15.79:0/1280583823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb39415d730 0x7fb3940cc2c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb39406cbe0 tx=0x7fb38c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cd3966a7b5c7f5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.740+0000 7fb3992e4700 1 -- 172.21.15.79:0/1280583823 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00e070 con 0x7fb39415d730 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb3992e4700 1 -- 172.21.15.79:0/1280583823 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c00b9c0 con 0x7fb39415d730 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb3992e4700 1 -- 172.21.15.79:0/1280583823 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c00bd70 con 0x7fb39415d730 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 -- 172.21.15.79:0/1280583823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb39415d730 msgr2=0x7fb3940cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 --2- 172.21.15.79:0/1280583823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb39415d730 0x7fb3940cc2c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb39406cbe0 tx=0x7fb38c00b330 comp rx=0 tx=0).stop 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 -- 172.21.15.79:0/1280583823 shutdown_connections 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 --2- 172.21.15.79:0/1280583823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb39415d730 0x7fb3940cc2c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 --2- 172.21.15.79:0/1280583823 >> v2:172.21.15.156:3300/0 conn(0x7fb394158e30 0x7fb3940ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 --2- 172.21.15.79:0/1280583823 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb394159200 0x7fb3940cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 -- 172.21.15.79:0/1280583823 >> 172.21.15.79:0/1280583823 conn(0x7fb3940b1d20 msgr2=0x7fb39406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:18.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 -- 172.21.15.79:0/1280583823 shutdown_connections 2023-06-08T03:51:18.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.741+0000 7fb39c54a700 1 -- 172.21.15.79:0/1280583823 wait complete. 2023-06-08T03:51:18.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.742+0000 7fb39c54a700 1 Processor -- start 2023-06-08T03:51:18.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.742+0000 7fb39c54a700 1 -- start start 2023-06-08T03:51:18.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39c54a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394158e30 0x7fb394147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39c54a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 0x7fb3940b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39c54a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb39415d730 0x7fb3940c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39c54a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940c8060 con 0x7fb394159200 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39c54a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3941ef550 con 0x7fb394158e30 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39c54a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3940c7730 con 0x7fb39415d730 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 0x7fb3940b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 0x7fb3940b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36364/0 (socket says 172.21.15.79:36364) 2023-06-08T03:51:18.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 -- 172.21.15.79:0/4002917813 learned_addr learned my addr 172.21.15.79:0/4002917813 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39aae7700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb39415d730 0x7fb3940c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb39a2e6700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394158e30 0x7fb394147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 -- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb39415d730 msgr2=0x7fb3940c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb39415d730 0x7fb3940c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 -- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394158e30 msgr2=0x7fb394147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394158e30 0x7fb394147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.743+0000 7fb399ae5700 1 -- 172.21.15.79:0/4002917813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c009b30 con 0x7fb394159200 2023-06-08T03:51:18.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.744+0000 7fb399ae5700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 0x7fb3940b4cb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c820 tx=0x7fb38400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.744+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384011800 con 0x7fb394159200 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.744+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384011960 con 0x7fb394159200 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.744+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384011d10 con 0x7fb394159200 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.744+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3940c79a0 con 0x7fb394159200 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.744+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3940b51f0 con 0x7fb394159200 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.745+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3940c81a0 con 0x7fb394159200 2023-06-08T03:51:18.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.745+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb384018360 con 0x7fb394159200 2023-06-08T03:51:18.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.746+0000 7fb38affd700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb37806ace0 0x7fb37806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:18.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.746+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb384089560 con 0x7fb394159200 2023-06-08T03:51:18.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.746+0000 7fb39a2e6700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb37806ace0 0x7fb37806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:18.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.746+0000 7fb39a2e6700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb37806ace0 0x7fb37806d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb390006fd0 tx=0x7fb390008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:18.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.750+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3940c81a0 con 0x7fb394159200 2023-06-08T03:51:18.941 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.937+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3940c81a0 con 0x7fb394159200 2023-06-08T03:51:18.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.939+0000 7fb38affd700 1 -- 172.21.15.79:0/4002917813 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fb3940c81a0 con 0x7fb394159200 2023-06-08T03:51:18.943 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:18.943 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:18.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb37806ace0 msgr2=0x7fb37806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb37806ace0 0x7fb37806d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb390006fd0 tx=0x7fb390008040 comp rx=0 tx=0).stop 2023-06-08T03:51:18.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 msgr2=0x7fb3940b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:18.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 0x7fb3940b4cb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb38400c820 tx=0x7fb38400cbe0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 shutdown_connections 2023-06-08T03:51:18.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb37806ace0 0x7fb37806d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb39415d730 0x7fb3940c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb394158e30 0x7fb394147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 --2- 172.21.15.79:0/4002917813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb394159200 0x7fb3940b4cb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:18.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 >> 172.21.15.79:0/4002917813 conn(0x7fb3940b1d20 msgr2=0x7fb3940af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:18.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 shutdown_connections 2023-06-08T03:51:18.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:18.941+0000 7fb39c54a700 1 -- 172.21.15.79:0/4002917813 wait complete. 2023-06-08T03:51:18.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:18.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:19.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.075+0000 7fb5d7d3a700 1 Processor -- start 2023-06-08T03:51:19.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.075+0000 7fb5d7d3a700 1 -- start start 2023-06-08T03:51:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d7d3a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb5d0158d50 0x7fb5d00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d7d3a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d015d3d0 0x7fb5d00cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d7d3a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb5d00cc4c0 0x7fb5d006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d7d3a700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d00cc920 con 0x7fb5d00cc4c0 2023-06-08T03:51:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d7d3a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d00cd7b0 con 0x7fb5d0158d50 2023-06-08T03:51:19.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d7d3a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00cda50 con 0x7fb5d015d3d0 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb5d00cc4c0 0x7fb5d006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb5d00cc4c0 0x7fb5d006eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36374/0 (socket says 172.21.15.79:36374) 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 -- 172.21.15.79:0/4183422491 learned_addr learned my addr 172.21.15.79:0/4183422491 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d5ad6700 1 --2- 172.21.15.79:0/4183422491 >> v2:172.21.15.156:3300/0 conn(0x7fb5d0158d50 0x7fb5d00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d52d5700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d015d3d0 0x7fb5d00cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 -- 172.21.15.79:0/4183422491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d015d3d0 msgr2=0x7fb5d00cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d015d3d0 0x7fb5d00cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 -- 172.21.15.79:0/4183422491 >> v2:172.21.15.156:3300/0 conn(0x7fb5d0158d50 msgr2=0x7fb5d00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 --2- 172.21.15.79:0/4183422491 >> v2:172.21.15.156:3300/0 conn(0x7fb5d0158d50 0x7fb5d00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 -- 172.21.15.79:0/4183422491 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00cdd30 con 0x7fb5d00cc4c0 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d52d5700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d015d3d0 0x7fb5d00cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d62d7700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d00cc4c0 0x7fb5d006eac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb5d006e5d0 tx=0x7fb5c800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64036802a769cd8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d4ad4700 1 -- 172.21.15.79:0/4183422491 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8010070 con 0x7fb5d00cc4c0 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d4ad4700 1 -- 172.21.15.79:0/4183422491 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c800dce0 con 0x7fb5d00cc4c0 2023-06-08T03:51:19.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.076+0000 7fb5d4ad4700 1 -- 172.21.15.79:0/4183422491 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c800b310 con 0x7fb5d00cc4c0 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/4183422491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d00cc4c0 msgr2=0x7fb5d006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d00cc4c0 0x7fb5d006eac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb5d006e5d0 tx=0x7fb5c800d430 comp rx=0 tx=0).stop 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/4183422491 shutdown_connections 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d00cc4c0 0x7fb5d006eac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/4183422491 >> v2:172.21.15.156:3300/0 conn(0x7fb5d0158d50 0x7fb5d00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/4183422491 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d015d3d0 0x7fb5d00cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/4183422491 >> 172.21.15.79:0/4183422491 conn(0x7fb5d00b1d30 msgr2=0x7fb5d006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:19.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/4183422491 shutdown_connections 2023-06-08T03:51:19.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.077+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/4183422491 wait complete. 2023-06-08T03:51:19.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.078+0000 7fb5d7d3a700 1 Processor -- start 2023-06-08T03:51:19.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.078+0000 7fb5d7d3a700 1 -- start start 2023-06-08T03:51:19.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d7d3a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d00cc4c0 0x7fb5d01489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d7d3a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d0158d50 0x7fb5d01f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d7d3a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 0x7fb5d01f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d7d3a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d0156700 con 0x7fb5d015d3d0 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d7d3a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d01513c0 con 0x7fb5d0158d50 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d7d3a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d01516a0 con 0x7fb5d00cc4c0 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 0x7fb5d01f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 0x7fb5d01f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36380/0 (socket says 172.21.15.79:36380) 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 -- 172.21.15.79:0/311730689 learned_addr learned my addr 172.21.15.79:0/311730689 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d5ad6700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d00cc4c0 0x7fb5d01489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d52d5700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d0158d50 0x7fb5d01f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 -- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d00cc4c0 msgr2=0x7fb5d01489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d00cc4c0 0x7fb5d01489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 -- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d0158d50 msgr2=0x7fb5d01f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d0158d50 0x7fb5d01f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 -- 172.21.15.79:0/311730689 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c800bd40 con 0x7fb5d015d3d0 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d5ad6700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d00cc4c0 0x7fb5d01489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d52d5700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d0158d50 0x7fb5d01f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:19.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.079+0000 7fb5d62d7700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 0x7fb5d01f5660 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0070720 tx=0x7fb5c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.080+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8009a10 con 0x7fb5d015d3d0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.080+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8009b90 con 0x7fb5d015d3d0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.080+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d0151980 con 0x7fb5d015d3d0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.080+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c800bbc0 con 0x7fb5d015d3d0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.080+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d0148ee0 con 0x7fb5d015d3d0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.081+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d0156870 con 0x7fb5d015d3d0 2023-06-08T03:51:19.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.081+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5c8004040 con 0x7fb5d015d3d0 2023-06-08T03:51:19.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.081+0000 7fb5c67fc700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb5b406ace0 0x7fb5b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.082+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb5c808ae80 con 0x7fb5d015d3d0 2023-06-08T03:51:19.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.082+0000 7fb5d5ad6700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb5b406ace0 0x7fb5b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.082+0000 7fb5d5ad6700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb5b406ace0 0x7fb5b406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009bb0 tx=0x7fb5cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.086+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5c8055cd0 con 0x7fb5d015d3d0 2023-06-08T03:51:19.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.274+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb5d0073e90 con 0x7fb5d015d3d0 2023-06-08T03:51:19.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.274+0000 7fb5c67fc700 1 -- 172.21.15.79:0/311730689 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb5c80592f0 con 0x7fb5d015d3d0 2023-06-08T03:51:19.278 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:19.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb5b406ace0 msgr2=0x7fb5b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb5b406ace0 0x7fb5b406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009bb0 tx=0x7fb5cc008040 comp rx=0 tx=0).stop 2023-06-08T03:51:19.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 msgr2=0x7fb5d01f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 0x7fb5d01f5660 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0070720 tx=0x7fb5c8004ad0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 shutdown_connections 2023-06-08T03:51:19.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb5d015d3d0 0x7fb5d01f5660 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb5b406ace0 0x7fb5b406d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb5d00cc4c0 0x7fb5d01489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 --2- 172.21.15.79:0/311730689 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d0158d50 0x7fb5d01f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 >> 172.21.15.79:0/311730689 conn(0x7fb5d00b1d30 msgr2=0x7fb5d00af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:19.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 shutdown_connections 2023-06-08T03:51:19.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.276+0000 7fb5d7d3a700 1 -- 172.21.15.79:0/311730689 wait complete. 2023-06-08T03:51:19.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:19.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.411+0000 7f302ad3a700 1 Processor -- start 2023-06-08T03:51:19.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.411+0000 7f302ad3a700 1 -- start start 2023-06-08T03:51:19.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f302ad3a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 0x7f302406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f302ad3a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30240cd140 con 0x7f3024069d50 2023-06-08T03:51:19.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f302ad3a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3024071110 con 0x7f3024071fe0 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f302ad3a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3024071be0 con 0x7f302415c1f0 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f30292d7700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3024069d50 0x7f30240cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59864/0 (socket says 172.21.15.79:59864) 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f30292d7700 1 -- 172.21.15.79:0/3143758054 learned_addr learned my addr 172.21.15.79:0/3143758054 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f3028ad6700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 0x7f302406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939211107 0 0) 0x7f30240cd140 con 0x7f3024069d50 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.412+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30140035b0 con 0x7f3024069d50 2023-06-08T03:51:19.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567915202 0 0) 0x7f30140035b0 con 0x7f3024069d50 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 msgr2=0x7f302406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 0x7f302406d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 >> v1:172.21.15.156:6789/0 conn(0x7f3024071fe0 legacy=0x7f30240cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240cd390 con 0x7f3024069d50 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f301c0019f0 con 0x7f3024069d50 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 >> v1:172.21.15.79:6789/0 conn(0x7f3024069d50 legacy=0x7f30240cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014003a30 0x7f3014003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3014004850 0x7f3014004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 0x7f3014005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3024071be0 con 0x7f3014004850 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301400bb40 con 0x7f3014003a30 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301400bd00 con 0x7f302415c1f0 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3014004850 0x7f3014004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3028ad6700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014003a30 0x7f3014003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30292d7700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 0x7f3014005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 -- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 msgr2=0x7f3014005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f302415c1f0 0x7f3014005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 -- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014003a30 msgr2=0x7f3014003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014003a30 0x7f3014003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 -- 172.21.15.79:0/3143758054 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140035b0 con 0x7f3014004850 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f3023fff700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3014004850 0x7f3014004cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f30100018d0 tx=0x7f3014005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f221ede6da8fc447 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.413+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010012620 con 0x7f3014004850 2023-06-08T03:51:19.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3010012780 con 0x7f3014004850 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f30237fe700 1 -- 172.21.15.79:0/3143758054 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010012b30 con 0x7f3014004850 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f302ad3a700 1 -- 172.21.15.79:0/3143758054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3014004850 msgr2=0x7f3014004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f302ad3a700 1 --2- 172.21.15.79:0/3143758054 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3014004850 0x7f3014004cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f30100018d0 tx=0x7f3014005880 comp rx=0 tx=0).stop 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f30292d7700 1 -- 172.21.15.79:0/3143758054 reap_dead start 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f302ad3a700 1 -- 172.21.15.79:0/3143758054 shutdown_connections 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f302ad3a700 1 -- 172.21.15.79:0/3143758054 >> 172.21.15.79:0/3143758054 conn(0x7f30240b1d30 msgr2=0x7f302406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:19.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f302ad3a700 1 -- 172.21.15.79:0/3143758054 shutdown_connections 2023-06-08T03:51:19.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.414+0000 7f302ad3a700 1 -- 172.21.15.79:0/3143758054 wait complete. 2023-06-08T03:51:19.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.415+0000 7f302ad3a700 1 Processor -- start 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.415+0000 7f302ad3a700 1 -- start start 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f302ad3a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014004850 0x7f30241f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f302ad3a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 0x7f30241e5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f302ad3a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30241e5890 0x7f30241f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f302ad3a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30241e5d10 con 0x7f30241f4ce0 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f302ad3a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240c7190 con 0x7f3014004850 2023-06-08T03:51:19.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f302ad3a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30241f27a0 con 0x7f30241e5890 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f3023fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 0x7f30241e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f3023fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 0x7f30241e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36398/0 (socket says 172.21.15.79:36398) 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f3023fff700 1 -- 172.21.15.79:0/2686626794 learned_addr learned my addr 172.21.15.79:0/2686626794 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f30292d7700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30241e5890 0x7f30241f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.416+0000 7f3028ad6700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014004850 0x7f30241f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3023fff700 1 -- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30241e5890 msgr2=0x7f30241f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3023fff700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30241e5890 0x7f30241f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3023fff700 1 -- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014004850 msgr2=0x7f30241f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3023fff700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014004850 0x7f30241f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3023fff700 1 -- 172.21.15.79:0/2686626794 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302415dbf0 con 0x7f30241f4ce0 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3028ad6700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014004850 0x7f30241f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f3023fff700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 0x7f30241e5350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f301400bb10 tx=0x7f3010001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30100052d0 con 0x7f30241f4ce0 2023-06-08T03:51:19.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3010005450 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302415ded0 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30100038c0 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.417+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30241f2a80 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.418+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3024073e90 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.418+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3010023070 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.419+0000 7f30217fa700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f300806ace0 0x7f300806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.419+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f301008ad70 con 0x7f30241f4ce0 2023-06-08T03:51:19.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.419+0000 7f3028ad6700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f300806ace0 0x7f300806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.419+0000 7f3028ad6700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f300806ace0 0x7f300806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3018009ae0 tx=0x7f3018008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.423+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3010055c40 con 0x7f30241f4ce0 2023-06-08T03:51:19.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.607+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f302406ed70 con 0x7f30241f4ce0 2023-06-08T03:51:19.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.607+0000 7f30217fa700 1 -- 172.21.15.79:0/2686626794 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f301002b020 con 0x7f30241f4ce0 2023-06-08T03:51:19.612 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f300806ace0 msgr2=0x7f300806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f300806ace0 0x7f300806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3018009ae0 tx=0x7f3018008040 comp rx=0 tx=0).stop 2023-06-08T03:51:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 msgr2=0x7f30241e5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 0x7f30241e5350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f301400bb10 tx=0x7f3010001230 comp rx=0 tx=0).stop 2023-06-08T03:51:19.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 shutdown_connections 2023-06-08T03:51:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f300806ace0 0x7f300806d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f30241e5890 0x7f30241f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3014004850 0x7f30241f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 --2- 172.21.15.79:0/2686626794 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f30241f4ce0 0x7f30241e5350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 >> 172.21.15.79:0/2686626794 conn(0x7f30240b1d30 msgr2=0x7f30240cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:19.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 shutdown_connections 2023-06-08T03:51:19.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.609+0000 7f302ad3a700 1 -- 172.21.15.79:0/2686626794 wait complete. 2023-06-08T03:51:19.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:19.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.744+0000 7f01ca039700 1 Processor -- start 2023-06-08T03:51:19.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.744+0000 7f01ca039700 1 -- start start 2023-06-08T03:51:19.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01ca039700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01c4163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01ca039700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40cd7b0 con 0x7f01c40ca360 2023-06-08T03:51:19.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01ca039700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40cda50 con 0x7f01c4158d50 2023-06-08T03:51:19.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01ca039700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40cdd30 con 0x7f01c40cc690 2023-06-08T03:51:19.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01c2ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f01c40ca360 0x7f01c40cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59880/0 (socket says 172.21.15.79:59880) 2023-06-08T03:51:19.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01c2ffd700 1 -- 172.21.15.79:0/3022820365 learned_addr learned my addr 172.21.15.79:0/3022820365 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:19.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508296920 0 0) 0x7f01c40cd7b0 con 0x7f01c40ca360 2023-06-08T03:51:19.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01c3fff700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01c4163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.745+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ac0035b0 con 0x7f01c40ca360 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1650397168 0 0) 0x7f01ac0035b0 con 0x7f01c40ca360 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 msgr2=0x7f01c4163d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01c4163d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 >> v1:172.21.15.156:6789/0 conn(0x7f01c4158d50 legacy=0x7f01c415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40ce010 con 0x7f01c40ca360 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f01b4001670 con 0x7f01c40ca360 2023-06-08T03:51:19.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 >> v1:172.21.15.79:6789/0 conn(0x7f01c40ca360 legacy=0x7f01c40cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac003a30 0x7f01ac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40cdd30 con 0x7f01ac004850 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac00bac0 con 0x7f01ac003a30 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac00bc50 con 0x7f01c40cc690 2023-06-08T03:51:19.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c37fe700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac003a30 0x7f01ac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c3fff700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 -- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 msgr2=0x7f01ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 -- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac003a30 msgr2=0x7f01ac003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac003a30 0x7f01ac003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 -- 172.21.15.79:0/3022820365 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac0035b0 con 0x7f01ac004850 2023-06-08T03:51:19.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c3fff700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40cc690 0x7f01ac005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:19.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01ac004cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c240 tx=0x7f01ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5b3ecf6386c268 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4007c10 con 0x7f01ac004850 2023-06-08T03:51:19.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b400cde0 con 0x7f01ac004850 2023-06-08T03:51:19.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.746+0000 7f01c27fc700 1 -- 172.21.15.79:0/3022820365 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4005ba0 con 0x7f01ac004850 2023-06-08T03:51:19.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01ca039700 1 -- 172.21.15.79:0/3022820365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 msgr2=0x7f01ac004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01ca039700 1 --2- 172.21.15.79:0/3022820365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01ac004cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c240 tx=0x7f01ac005880 comp rx=0 tx=0).stop 2023-06-08T03:51:19.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01c3fff700 1 -- 172.21.15.79:0/3022820365 reap_dead start 2023-06-08T03:51:19.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01ca039700 1 -- 172.21.15.79:0/3022820365 shutdown_connections 2023-06-08T03:51:19.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01ca039700 1 -- 172.21.15.79:0/3022820365 >> 172.21.15.79:0/3022820365 conn(0x7f01c40b1d30 msgr2=0x7f01c406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:19.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01ca039700 1 -- 172.21.15.79:0/3022820365 shutdown_connections 2023-06-08T03:51:19.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.747+0000 7f01ca039700 1 -- 172.21.15.79:0/3022820365 wait complete. 2023-06-08T03:51:19.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.748+0000 7f01ca039700 1 Processor -- start 2023-06-08T03:51:19.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 -- start start 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01c41580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c414d7d0 0x7f01c406a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40c5110 0x7f01c41443c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c406a1d0 con 0x7f01ac004850 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40b4010 con 0x7f01c414d7d0 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01ca039700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4152ed0 con 0x7f01c40c5110 2023-06-08T03:51:19.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01c41580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01c41580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36416/0 (socket says 172.21.15.79:36416) 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.749+0000 7f01c37fe700 1 -- 172.21.15.79:0/2743366821 learned_addr learned my addr 172.21.15.79:0/2743366821 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c414d7d0 0x7f01c406a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c3fff700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40c5110 0x7f01c41443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c37fe700 1 -- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40c5110 msgr2=0x7f01c41443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c37fe700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40c5110 0x7f01c41443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c37fe700 1 -- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c414d7d0 msgr2=0x7f01c406a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c37fe700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c414d7d0 0x7f01c406a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c37fe700 1 -- 172.21.15.79:0/2743366821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c406ab30 con 0x7f01ac004850 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c3fff700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40c5110 0x7f01c41443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01c37fe700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01c41580c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f01b800f820 tx=0x7f01b800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8005320 con 0x7f01ac004850 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b8005480 con 0x7f01ac004850 2023-06-08T03:51:19.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b8006aa0 con 0x7f01ac004850 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c4152910 con 0x7f01ac004850 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.750+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c4069df0 con 0x7f01ac004850 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.751+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4073e90 con 0x7f01ac004850 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.751+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f01b8006c00 con 0x7f01ac004850 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.752+0000 7f01abfff700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f01b006ace0 0x7f01b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.752+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f01b8088d70 con 0x7f01ac004850 2023-06-08T03:51:19.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.752+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f01b006ace0 0x7f01b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:19.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.752+0000 7f01c2ffd700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f01b006ace0 0x7f01b006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f01c4071300 tx=0x7f01b4007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:19.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.756+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f01b8053ab0 con 0x7f01ac004850 2023-06-08T03:51:19.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.941+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01c406d9a0 con 0x7f01ac004850 2023-06-08T03:51:19.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.942+0000 7f01abfff700 1 -- 172.21.15.79:0/2743366821 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01b80570d0 con 0x7f01ac004850 2023-06-08T03:51:19.946 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f01b006ace0 msgr2=0x7f01b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f01b006ace0 0x7f01b006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f01c4071300 tx=0x7f01b4007000 comp rx=0 tx=0).stop 2023-06-08T03:51:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 msgr2=0x7f01c41580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01c41580c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f01b800f820 tx=0x7f01b800fbe0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.948 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 shutdown_connections 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f01b006ace0 0x7f01b006d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c40c5110 0x7f01c41443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01ac004850 0x7f01c41580c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 --2- 172.21.15.79:0/2743366821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c414d7d0 0x7f01c406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 >> 172.21.15.79:0/2743366821 conn(0x7f01c40b1d30 msgr2=0x7f01c40cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 shutdown_connections 2023-06-08T03:51:19.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:19.944+0000 7f01ca039700 1 -- 172.21.15.79:0/2743366821 wait complete. 2023-06-08T03:51:19.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:20.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.077+0000 7f8f40bba700 1 Processor -- start 2023-06-08T03:51:20.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.077+0000 7f8f40bba700 1 -- start start 2023-06-08T03:51:20.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f40bba700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8f3c15a0e0 0x7f8f3c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f40bba700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c0707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f40bba700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c071290 con 0x7f8f3c070d10 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f40bba700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c071570 con 0x7f8f3c15a0e0 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f40bba700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0ca4c0 con 0x7f8f3c06a290 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f3a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8f3c15a0e0 0x7f8f3c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.078+0000 7f8f3a59c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8f3c15a0e0 0x7f8f3c069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36424/0 (socket says 172.21.15.79:36424) 2023-06-08T03:51:20.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3a59c700 1 -- 172.21.15.79:0/824209602 learned_addr learned my addr 172.21.15.79:0/824209602 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f39d9b700 1 --2- 172.21.15.79:0/824209602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c0707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3a59c700 1 -- 172.21.15.79:0/824209602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 msgr2=0x7f8f3c0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3a59c700 1 --2- 172.21.15.79:0/824209602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c0707d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3a59c700 1 -- 172.21.15.79:0/824209602 >> v1:172.21.15.156:6789/0 conn(0x7f8f3c070d10 legacy=0x7f8f3c0cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3a59c700 1 -- 172.21.15.79:0/824209602 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0ca7a0 con 0x7f8f3c15a0e0 2023-06-08T03:51:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3a59c700 1 --2- 172.21.15.79:0/824209602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c069d50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0098f0 tx=0x7f8f2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f19e69e3e8026df server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3959a700 1 -- 172.21.15.79:0/824209602 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c011830 con 0x7f8f3c15a0e0 2023-06-08T03:51:20.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3959a700 1 -- 172.21.15.79:0/824209602 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c011990 con 0x7f8f3c15a0e0 2023-06-08T03:51:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.079+0000 7f8f3959a700 1 -- 172.21.15.79:0/824209602 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c011d20 con 0x7f8f3c15a0e0 2023-06-08T03:51:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 -- 172.21.15.79:0/824209602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c15a0e0 msgr2=0x7f8f3c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 --2- 172.21.15.79:0/824209602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c069d50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0098f0 tx=0x7f8f2c009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 -- 172.21.15.79:0/824209602 shutdown_connections 2023-06-08T03:51:20.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 --2- 172.21.15.79:0/824209602 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c069d50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 --2- 172.21.15.79:0/824209602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c0707d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 -- 172.21.15.79:0/824209602 >> 172.21.15.79:0/824209602 conn(0x7f8f3c0ced30 msgr2=0x7f8f3c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 -- 172.21.15.79:0/824209602 shutdown_connections 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.080+0000 7f8f40bba700 1 -- 172.21.15.79:0/824209602 wait complete. 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.081+0000 7f8f40bba700 1 Processor -- start 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.081+0000 7f8f40bba700 1 -- start start 2023-06-08T03:51:20.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 0x7f8f3c0c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c0c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c1560e0 con 0x7f8f3c070d10 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0b34f0 con 0x7f8f3c15a0e0 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0c4660 con 0x7f8f3c06a290 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 0x7f8f3c0c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 0x7f8f3c0c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36432/0 (socket says 172.21.15.79:36432) 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 -- 172.21.15.79:0/2785233889 learned_addr learned my addr 172.21.15.79:0/2785233889 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f3a59c700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f3ad9d700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c0c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 -- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 msgr2=0x7f8f3c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 -- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15a0e0 msgr2=0x7f8f3c0c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c0c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 -- 172.21.15.79:0/2785233889 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c009550 con 0x7f8f3c070d10 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f3a59c700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f3ad9d700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f39d9b700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 0x7f8f3c0c4120 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e870 tx=0x7f8f3000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30009590 con 0x7f8f3c070d10 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3000ee70 con 0x7f8f3c070d10 2023-06-08T03:51:20.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c0b28e0 con 0x7f8f3c070d10 2023-06-08T03:51:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.082+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3000ca50 con 0x7f8f3c070d10 2023-06-08T03:51:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.083+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c0b2ea0 con 0x7f8f3c070d10 2023-06-08T03:51:20.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.083+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c06f1d0 con 0x7f8f3c070d10 2023-06-08T03:51:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.084+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f3000cbb0 con 0x7f8f3c070d10 2023-06-08T03:51:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.084+0000 7f8f2affd700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1c06ace0 0x7f8f1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.084+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8f300896b0 con 0x7f8f3c070d10 2023-06-08T03:51:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.085+0000 7f8f3a59c700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1c06ace0 0x7f8f1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.085+0000 7f8f3a59c700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1c06ace0 0x7f8f1c06d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009c40 tx=0x7f8f2c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.088+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f30057f90 con 0x7f8f3c070d10 2023-06-08T03:51:20.278 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.274+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f3c0b3660 con 0x7f8f3c070d10 2023-06-08T03:51:20.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.275+0000 7f8f2affd700 1 -- 172.21.15.79:0/2785233889 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f3000ce60 con 0x7f8f3c070d10 2023-06-08T03:51:20.279 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:20.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1c06ace0 msgr2=0x7f8f1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1c06ace0 0x7f8f1c06d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009c40 tx=0x7f8f2c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 msgr2=0x7f8f3c0c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 0x7f8f3c0c4120 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000e870 tx=0x7f8f3000ec30 comp rx=0 tx=0).stop 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 shutdown_connections 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f3c15a0e0 0x7f8f3c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8f1c06ace0 0x7f8f1c06d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8f3c06a290 0x7f8f3c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.282 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 --2- 172.21.15.79:0/2785233889 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f3c070d10 0x7f8f3c0c4120 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 >> 172.21.15.79:0/2785233889 conn(0x7f8f3c0ced30 msgr2=0x7f8f3c15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:20.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 shutdown_connections 2023-06-08T03:51:20.283 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.277+0000 7f8f40bba700 1 -- 172.21.15.79:0/2785233889 wait complete. 2023-06-08T03:51:20.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:20.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.413+0000 7f6eed44d700 1 Processor -- start 2023-06-08T03:51:20.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.413+0000 7f6eed44d700 1 -- start start 2023-06-08T03:51:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6eed44d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6ee80ccef0 0x7f6ee80d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6eed44d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80b5940 con 0x7f6ee80d09b0 2023-06-08T03:51:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6eed44d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80cef00 con 0x7f6ee815f820 2023-06-08T03:51:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6eed44d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80d0dd0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee67fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6ee80ccef0 0x7f6ee80d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6ee80d09b0 0x7f6ee80b5dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59916/0 (socket says 172.21.15.79:59916) 2023-06-08T03:51:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee77fe700 1 -- 172.21.15.79:0/2617667619 learned_addr learned my addr 172.21.15.79:0/2617667619 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:20.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee67fc700 1 -- 172.21.15.79:0/2617667619 >> v1:172.21.15.156:6789/0 conn(0x7f6ee815f820 legacy=0x7f6ee80cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee67fc700 1 -- 172.21.15.79:0/2617667619 >> v1:172.21.15.79:6789/0 conn(0x7f6ee80d09b0 legacy=0x7f6ee80b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee67fc700 1 -- 172.21.15.79:0/2617667619 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee80ce660 con 0x7f6ee80ccef0 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee5ffb700 1 -- 172.21.15.79:0/2617667619 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960262882 0 0) 0x7f6ee80b5940 con 0x7f6ee80d09b0 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee67fc700 1 --2- 172.21.15.79:0/2617667619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee80d0470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009840 tx=0x7f6ed8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a000c6f8a6209a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee5ffb700 1 -- 172.21.15.79:0/2617667619 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8004230 con 0x7f6ee80ccef0 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee5ffb700 1 -- 172.21.15.79:0/2617667619 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed8004390 con 0x7f6ee80ccef0 2023-06-08T03:51:20.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.414+0000 7f6ee5ffb700 1 -- 172.21.15.79:0/2617667619 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80036a0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 -- 172.21.15.79:0/2617667619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 msgr2=0x7f6ee80d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 --2- 172.21.15.79:0/2617667619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee80d0470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009840 tx=0x7f6ed8009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:20.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 -- 172.21.15.79:0/2617667619 shutdown_connections 2023-06-08T03:51:20.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 --2- 172.21.15.79:0/2617667619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee80d0470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 -- 172.21.15.79:0/2617667619 >> 172.21.15.79:0/2617667619 conn(0x7f6ee809bc40 msgr2=0x7f6ee80d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:20.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 -- 172.21.15.79:0/2617667619 shutdown_connections 2023-06-08T03:51:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.415+0000 7f6eed44d700 1 -- 172.21.15.79:0/2617667619 wait complete. 2023-06-08T03:51:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.416+0000 7f6eed44d700 1 Processor -- start 2023-06-08T03:51:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.416+0000 7f6eed44d700 1 -- start start 2023-06-08T03:51:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6eed44d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee8151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6eed44d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80d09b0 0x7f6ee80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6eed44d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815f820 0x7f6ee80b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6eed44d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80c9fc0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6eed44d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80a01b0 con 0x7f6ee815f820 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6eed44d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80a0490 con 0x7f6ee80d09b0 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee8151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee8151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36444/0 (socket says 172.21.15.79:36444) 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 -- 172.21.15.79:0/798453422 learned_addr learned my addr 172.21.15.79:0/798453422 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:20.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee67fc700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80d09b0 0x7f6ee80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee77fe700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815f820 0x7f6ee80b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 -- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80d09b0 msgr2=0x7f6ee80b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80d09b0 0x7f6ee80b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 -- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815f820 msgr2=0x7f6ee80b6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815f820 0x7f6ee80b6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee6ffd700 1 -- 172.21.15.79:0/798453422 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80094a0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee67fc700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80d09b0 0x7f6ee80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.417+0000 7f6ee77fe700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815f820 0x7f6ee80b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.418+0000 7f6ee6ffd700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee8151e30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c520 tx=0x7f6edc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.418+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0054e0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.418+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0036d0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.418+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee80c9970 con 0x7f6ee80ccef0 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.418+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc003a80 con 0x7f6ee80ccef0 2023-06-08T03:51:20.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.418+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee80b7cf0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.419+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8073e90 con 0x7f6ee80ccef0 2023-06-08T03:51:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.419+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6edc003be0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.419+0000 7f6ecf7fe700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6ed406ace0 0x7f6ed406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.420+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6edc088c10 con 0x7f6ee80ccef0 2023-06-08T03:51:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.420+0000 7f6ee67fc700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6ed406ace0 0x7f6ed406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.420+0000 7f6ee67fc700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6ed406ace0 0x7f6ed406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ee806b120 tx=0x7f6ed8013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.424+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6edc053a60 con 0x7f6ee80ccef0 2023-06-08T03:51:20.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.607+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6ee80b6300 con 0x7f6ee80ccef0 2023-06-08T03:51:20.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.607+0000 7f6ecf7fe700 1 -- 172.21.15.79:0/798453422 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6edc0170a0 con 0x7f6ee80ccef0 2023-06-08T03:51:20.612 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:20.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.609+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6ed406ace0 msgr2=0x7f6ed406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.609+0000 7f6eed44d700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6ed406ace0 0x7f6ed406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ee806b120 tx=0x7f6ed8013040 comp rx=0 tx=0).stop 2023-06-08T03:51:20.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.609+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 msgr2=0x7f6ee8151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.609+0000 7f6eed44d700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee8151e30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c520 tx=0x7f6edc00c8e0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.609+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 shutdown_connections 2023-06-08T03:51:20.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.609+0000 7f6eed44d700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815f820 0x7f6ee80b6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.610+0000 7f6eed44d700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ee80ccef0 0x7f6ee8151e30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.610+0000 7f6eed44d700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6ed406ace0 0x7f6ed406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.610+0000 7f6eed44d700 1 --2- 172.21.15.79:0/798453422 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80d09b0 0x7f6ee80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.610+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 >> 172.21.15.79:0/798453422 conn(0x7f6ee809bc40 msgr2=0x7f6ee81e5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:20.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.610+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 shutdown_connections 2023-06-08T03:51:20.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.610+0000 7f6eed44d700 1 -- 172.21.15.79:0/798453422 wait complete. 2023-06-08T03:51:20.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:20.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.745+0000 7f5464727700 1 Processor -- start 2023-06-08T03:51:20.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.745+0000 7f5464727700 1 -- start start 2023-06-08T03:51:20.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5464727700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f545c0ca360 0x7f545c0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5464727700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0cc900 con 0x7f545c0cc4c0 2023-06-08T03:51:20.751 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5464727700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0cd7b0 con 0x7f545c158d50 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5464727700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c0cda50 con 0x7f545c0ca360 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5461cc2700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f545c0ca360 0x7f545c0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5462cc4700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f545c0cc4c0 0x7f545c071220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59932/0 (socket says 172.21.15.79:59932) 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5462cc4700 1 -- 172.21.15.79:0/173077338 learned_addr learned my addr 172.21.15.79:0/173077338 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5461cc2700 1 -- 172.21.15.79:0/173077338 >> v1:172.21.15.156:6789/0 conn(0x7f545c158d50 legacy=0x7f545c15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5461cc2700 1 -- 172.21.15.79:0/173077338 >> v1:172.21.15.79:6789/0 conn(0x7f545c0cc4c0 legacy=0x7f545c071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f54614c1700 1 -- 172.21.15.79:0/173077338 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227720182 0 0) 0x7f545c0cc900 con 0x7f545c0cc4c0 2023-06-08T03:51:20.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5461cc2700 1 -- 172.21.15.79:0/173077338 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0cdd30 con 0x7f545c0ca360 2023-06-08T03:51:20.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f5461cc2700 1 --2- 172.21.15.79:0/173077338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0ca360 0x7f545c0cbf80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f545800bae0 tx=0x7f545800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6857d99f1dfdb1c server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.747+0000 7f54614c1700 1 -- 172.21.15.79:0/173077338 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545800dd30 con 0x7f545c0ca360 2023-06-08T03:51:20.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f54614c1700 1 -- 172.21.15.79:0/173077338 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545800de90 con 0x7f545c0ca360 2023-06-08T03:51:20.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f54614c1700 1 -- 172.21.15.79:0/173077338 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458004250 con 0x7f545c0ca360 2023-06-08T03:51:20.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 -- 172.21.15.79:0/173077338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0ca360 msgr2=0x7f545c0cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 --2- 172.21.15.79:0/173077338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0ca360 0x7f545c0cbf80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f545800bae0 tx=0x7f545800bdf0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 -- 172.21.15.79:0/173077338 shutdown_connections 2023-06-08T03:51:20.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 --2- 172.21.15.79:0/173077338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0ca360 0x7f545c0cbf80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 -- 172.21.15.79:0/173077338 >> 172.21.15.79:0/173077338 conn(0x7f545c0b1d30 msgr2=0x7f545c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 -- 172.21.15.79:0/173077338 shutdown_connections 2023-06-08T03:51:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.748+0000 7f5464727700 1 -- 172.21.15.79:0/173077338 wait complete. 2023-06-08T03:51:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.750+0000 7f5464727700 1 Processor -- start 2023-06-08T03:51:20.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.750+0000 7f5464727700 1 -- start start 2023-06-08T03:51:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f545c0ca360 0x7f545c14cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 0x7f545c0c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c158d50 0x7f545c0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c1567a0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c151cf0 con 0x7f545c158d50 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c151fd0 con 0x7f545c0ca360 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 0x7f545c0c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 0x7f545c0c2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36460/0 (socket says 172.21.15.79:36460) 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 -- 172.21.15.79:0/2647714258 learned_addr learned my addr 172.21.15.79:0/2647714258 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f54624c3700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f545c0ca360 0x7f545c14cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5462cc4700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c158d50 0x7f545c0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 -- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f545c0ca360 msgr2=0x7f545c14cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f545c0ca360 0x7f545c14cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 -- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c158d50 msgr2=0x7f545c0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c158d50 0x7f545c0c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 -- 172.21.15.79:0/2647714258 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545800b740 con 0x7f545c0cc4c0 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f54624c3700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f545c0ca360 0x7f545c14cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5462cc4700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c158d50 0x7f545c0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5461cc2700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 0x7f545c0c2f10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f545800b2c0 tx=0x7f5458004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.759 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458003dd0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545801f040 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c1522b0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580106b0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.751+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c14d040 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.752+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c073e90 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.753+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f545802c410 con 0x7f545c0cc4c0 2023-06-08T03:51:20.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.753+0000 7f5452ffd700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f544006ad10 0x7f544006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.753+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f545808d3a0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.753+0000 7f54624c3700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f544006ad10 0x7f544006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.754+0000 7f54624c3700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f544006ad10 0x7f544006d1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:20.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.757+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f54580581f0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.962+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f545c06d9a0 con 0x7f545c0cc4c0 2023-06-08T03:51:20.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.963+0000 7f5452ffd700 1 -- 172.21.15.79:0/2647714258 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f545805b810 con 0x7f545c0cc4c0 2023-06-08T03:51:20.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f544006ad10 msgr2=0x7f544006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f544006ad10 0x7f544006d1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c008040 comp rx=0 tx=0).stop 2023-06-08T03:51:20.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 msgr2=0x7f545c0c2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:20.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 0x7f545c0c2f10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f545800b2c0 tx=0x7f5458004750 comp rx=0 tx=0).stop 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 shutdown_connections 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c158d50 0x7f545c0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f544006ad10 0x7f544006d1c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f545c0ca360 0x7f545c14cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 --2- 172.21.15.79:0/2647714258 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f545c0cc4c0 0x7f545c0c2f10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 >> 172.21.15.79:0/2647714258 conn(0x7f545c0b1d30 msgr2=0x7f545c0cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 shutdown_connections 2023-06-08T03:51:20.970 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:20.965+0000 7f5464727700 1 -- 172.21.15.79:0/2647714258 wait complete. 2023-06-08T03:51:20.971 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:20.981 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:51:20.981 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:21.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:21.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.118+0000 7f5019a65700 1 Processor -- start 2023-06-08T03:51:21.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.118+0000 7f5019a65700 1 -- start start 2023-06-08T03:51:21.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.118+0000 7f5019a65700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 0x7f5014071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5019a65700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50140cee20 con 0x7f50140cd6c0 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5019a65700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f501406fa00 con 0x7f501415f6c0 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5019a65700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cf330 con 0x7f501406a460 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5013fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f50140cd6c0 0x7f5014071590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59946/0 (socket says 172.21.15.79:59946) 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5013fff700 1 -- 172.21.15.79:0/129311386 learned_addr learned my addr 172.21.15.79:0/129311386 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5012ffd700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 0x7f5014071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424158579 0 0) 0x7f50140cee20 con 0x7f50140cd6c0 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ffc0035b0 con 0x7f50140cd6c0 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 262959225 0 0) 0x7f4ffc0035b0 con 0x7f50140cd6c0 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 msgr2=0x7f5014071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 0x7f5014071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 >> v1:172.21.15.156:6789/0 conn(0x7f501415f6c0 legacy=0x7f5014069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50140cd1e0 con 0x7f50140cd6c0 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f500c0019f0 con 0x7f50140cd6c0 2023-06-08T03:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 >> v1:172.21.15.79:6789/0 conn(0x7f50140cd6c0 legacy=0x7f5014071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ffc003aa0 0x7f4ffc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 0x7f4ffc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ffc004d90 0x7f4ffc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50140cf330 con 0x7f4ffc004d90 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc00bac0 con 0x7f4ffc003aa0 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc00bc80 con 0x7f501406a460 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5013fff700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ffc004d90 0x7f4ffc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f50137fe700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ffc003aa0 0x7f4ffc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.119+0000 7f5012ffd700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 0x7f4ffc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f5013fff700 1 -- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 msgr2=0x7f4ffc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f5013fff700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501406a460 0x7f4ffc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f5013fff700 1 -- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ffc003aa0 msgr2=0x7f4ffc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f5013fff700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ffc003aa0 0x7f4ffc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f5013fff700 1 -- 172.21.15.79:0/129311386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0035b0 con 0x7f4ffc004d90 2023-06-08T03:51:21.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f5013fff700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ffc004d90 0x7f4ffc005200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f50140cdb00 tx=0x7f4ffc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f57cc0f7ccf59f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c00a300 con 0x7f4ffc004d90 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c012620 con 0x7f4ffc004d90 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.120+0000 7f50127fc700 1 -- 172.21.15.79:0/129311386 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c0129d0 con 0x7f4ffc004d90 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5019a65700 1 -- 172.21.15.79:0/129311386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ffc004d90 msgr2=0x7f4ffc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5019a65700 1 --2- 172.21.15.79:0/129311386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4ffc004d90 0x7f4ffc005200 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f50140cdb00 tx=0x7f4ffc005880 comp rx=0 tx=0).stop 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5013fff700 1 -- 172.21.15.79:0/129311386 reap_dead start 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5019a65700 1 -- 172.21.15.79:0/129311386 shutdown_connections 2023-06-08T03:51:21.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5019a65700 1 -- 172.21.15.79:0/129311386 >> 172.21.15.79:0/129311386 conn(0x7f50140b1d20 msgr2=0x7f501406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:21.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5019a65700 1 -- 172.21.15.79:0/129311386 shutdown_connections 2023-06-08T03:51:21.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.121+0000 7f5019a65700 1 -- 172.21.15.79:0/129311386 wait complete. 2023-06-08T03:51:21.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.122+0000 7f5019a65700 1 Processor -- start 2023-06-08T03:51:21.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.122+0000 7f5019a65700 1 -- start start 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5019a65700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 0x7f501414f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5019a65700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f501414c280 0x7f501414d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5019a65700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501414fcb0 0x7f501414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5019a65700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501414c740 con 0x7f501414bc90 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5019a65700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501414ca20 con 0x7f501414c280 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5019a65700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501414cd00 con 0x7f501414fcb0 2023-06-08T03:51:21.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 0x7f501414f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 0x7f501414f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36478/0 (socket says 172.21.15.79:36478) 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 -- 172.21.15.79:0/3611140676 learned_addr learned my addr 172.21.15.79:0/3611140676 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5013fff700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501414fcb0 0x7f501414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5012ffd700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f501414c280 0x7f501414d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 -- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501414fcb0 msgr2=0x7f501414dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501414fcb0 0x7f501414dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 -- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f501414c280 msgr2=0x7f501414d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f501414c280 0x7f501414d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f50137fe700 1 -- 172.21.15.79:0/3611140676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501414cfe0 con 0x7f501414bc90 2023-06-08T03:51:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.123+0000 7f5012ffd700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f501414c280 0x7f501414d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.124+0000 7f50137fe700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 0x7f501414f770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f500400f860 tx=0x7f500400fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.127+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500400fe60 con 0x7f501414bc90 2023-06-08T03:51:21.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.127+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501414d2c0 con 0x7f501414bc90 2023-06-08T03:51:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.127+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50140ac910 con 0x7f501414bc90 2023-06-08T03:51:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.127+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50040052e0 con 0x7f501414bc90 2023-06-08T03:51:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.127+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004003680 con 0x7f501414bc90 2023-06-08T03:51:21.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.128+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5014073e90 con 0x7f501414bc90 2023-06-08T03:51:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.129+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50040037e0 con 0x7f501414bc90 2023-06-08T03:51:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.129+0000 7f4ffbfff700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f500006ace0 0x7f500006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.129+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5004089760 con 0x7f501414bc90 2023-06-08T03:51:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.129+0000 7f5012ffd700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f500006ace0 0x7f500006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.130+0000 7f5012ffd700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f500006ace0 0x7f500006d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f501414c670 tx=0x7f5008008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.133+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f50040544a0 con 0x7f501414bc90 2023-06-08T03:51:21.323 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.319+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50140af970 con 0x7f501414bc90 2023-06-08T03:51:21.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.320+0000 7f4ffbfff700 1 -- 172.21.15.79:0/3611140676 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f5004057ac0 con 0x7f501414bc90 2023-06-08T03:51:21.324 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:21.325 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:21.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f500006ace0 msgr2=0x7f500006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f500006ace0 0x7f500006d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f501414c670 tx=0x7f5008008040 comp rx=0 tx=0).stop 2023-06-08T03:51:21.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 msgr2=0x7f501414f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 0x7f501414f770 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f500400f860 tx=0x7f500400fc20 comp rx=0 tx=0).stop 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 shutdown_connections 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f500006ace0 0x7f500006d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f501414fcb0 0x7f501414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f501414bc90 0x7f501414f770 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 --2- 172.21.15.79:0/3611140676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f501414c280 0x7f501414d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 >> 172.21.15.79:0/3611140676 conn(0x7f50140b1d20 msgr2=0x7f501415eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:21.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 shutdown_connections 2023-06-08T03:51:21.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.322+0000 7f5019a65700 1 -- 172.21.15.79:0/3611140676 wait complete. 2023-06-08T03:51:21.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:21.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:21.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 Processor -- start 2023-06-08T03:51:21.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 -- start start 2023-06-08T03:51:21.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 0x7f698406cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f698406d480 0x7f69840c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69840ce330 con 0x7f69840c9070 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840c94f0 con 0x7f698406d480 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.457+0000 7f6989832700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840abaa0 con 0x7f69840cdf60 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f69837fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f69840c9070 0x7f698415c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59964/0 (socket says 172.21.15.79:59964) 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f69837fe700 1 -- 172.21.15.79:0/717474107 learned_addr learned my addr 172.21.15.79:0/717474107 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:21.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6982ffd700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 0x7f698406cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f69827fc700 1 --2- 172.21.15.79:0/717474107 >> v2:172.21.15.156:3300/0 conn(0x7f698406d480 0x7f69840c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8628503 0 0) 0x7f69840ce330 con 0x7f69840c9070 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69700035b0 con 0x7f69840c9070 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95817669 0 0) 0x7f69700035b0 con 0x7f69840c9070 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 msgr2=0x7f698406cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 0x7f698406cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 >> v2:172.21.15.156:3300/0 conn(0x7f698406d480 msgr2=0x7f69840c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 --2- 172.21.15.79:0/717474107 >> v2:172.21.15.156:3300/0 conn(0x7f698406d480 0x7f69840c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840acf40 con 0x7f69840c9070 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f697c0019f0 con 0x7f69840c9070 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 >> v1:172.21.15.79:6789/0 conn(0x7f69840c9070 legacy=0x7f698415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6970003a00 0x7f6970003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 0x7f6970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6970004d90 0x7f6970005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840c94f0 con 0x7f6970003a00 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840abaa0 con 0x7f6970004d90 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697000ba10 con 0x7f69840cdf60 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f6982ffd700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6970003a00 0x7f6970003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f69827fc700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 0x7f6970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.458+0000 7f69837fe700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6970004d90 0x7f6970005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6982ffd700 1 -- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 msgr2=0x7f6970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6982ffd700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f69840cdf60 0x7f6970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6982ffd700 1 -- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6970004d90 msgr2=0x7f6970005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6982ffd700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6970004d90 0x7f6970005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6982ffd700 1 -- 172.21.15.79:0/717474107 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700035b0 con 0x7f6970003a00 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f69837fe700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6970004d90 0x7f6970005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6982ffd700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6970003a00 0x7f6970003e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f697400be60 tx=0x7f6970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe3b094bb2dac6f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974012040 con 0x7f6970003a00 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697400ddc0 con 0x7f6970003a00 2023-06-08T03:51:21.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6981ffb700 1 -- 172.21.15.79:0/717474107 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740185f0 con 0x7f6970003a00 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6989832700 1 -- 172.21.15.79:0/717474107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6970003a00 msgr2=0x7f6970003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6989832700 1 --2- 172.21.15.79:0/717474107 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6970003a00 0x7f6970003e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f697400be60 tx=0x7f6970005880 comp rx=0 tx=0).stop 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f69837fe700 1 -- 172.21.15.79:0/717474107 reap_dead start 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6989832700 1 -- 172.21.15.79:0/717474107 shutdown_connections 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6989832700 1 -- 172.21.15.79:0/717474107 >> 172.21.15.79:0/717474107 conn(0x7f69840ce4e0 msgr2=0x7f69840ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6989832700 1 -- 172.21.15.79:0/717474107 shutdown_connections 2023-06-08T03:51:21.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.459+0000 7f6989832700 1 -- 172.21.15.79:0/717474107 wait complete. 2023-06-08T03:51:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.461+0000 7f6989832700 1 Processor -- start 2023-06-08T03:51:21.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.461+0000 7f6989832700 1 -- start start 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6989832700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6970003a00 0x7f69841564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6989832700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 0x7f6984157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6989832700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69840c7e00 0x7f69840c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6989832700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840978b0 con 0x7f69841579e0 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6989832700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b2500 con 0x7f69840c7e00 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6989832700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6984158390 con 0x7f6970003a00 2023-06-08T03:51:21.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 0x7f6984157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 0x7f6984157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36498/0 (socket says 172.21.15.79:36498) 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 -- 172.21.15.79:0/1231540493 learned_addr learned my addr 172.21.15.79:0/1231540493 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f6982ffd700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6970003a00 0x7f69841564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69837fe700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69840c7e00 0x7f69840c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 -- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6970003a00 msgr2=0x7f69841564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6970003a00 0x7f69841564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 -- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69840c7e00 msgr2=0x7f69840c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69840c7e00 0x7f69840c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.462+0000 7f69827fc700 1 -- 172.21.15.79:0/1231540493 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6984156a30 con 0x7f69841579e0 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.463+0000 7f69827fc700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 0x7f6984157e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f696c011b00 tx=0x7f696c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.463+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00fa00 con 0x7f69841579e0 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.463+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00fb60 con 0x7f69841579e0 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.463+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69840c87b0 con 0x7f69841579e0 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.463+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c014440 con 0x7f69841579e0 2023-06-08T03:51:21.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.463+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69840b1b60 con 0x7f69841579e0 2023-06-08T03:51:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.464+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6984073e90 con 0x7f69841579e0 2023-06-08T03:51:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.464+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f696c0145a0 con 0x7f69841579e0 2023-06-08T03:51:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.464+0000 7f697bfff700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f696406ace0 0x7f696406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.464+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f696c089f30 con 0x7f69841579e0 2023-06-08T03:51:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.465+0000 7f6982ffd700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f696406ace0 0x7f696406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.465+0000 7f6982ffd700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f696406ace0 0x7f696406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f697400bd80 tx=0x7f6974013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.469+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f696c054c60 con 0x7f69841579e0 2023-06-08T03:51:21.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.651+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f69840cbb70 con 0x7f69841579e0 2023-06-08T03:51:21.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.652+0000 7f697bfff700 1 -- 172.21.15.79:0/1231540493 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f696c058280 con 0x7f69841579e0 2023-06-08T03:51:21.656 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:21.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.653+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f696406ace0 msgr2=0x7f696406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.653+0000 7f6989832700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f696406ace0 0x7f696406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f697400bd80 tx=0x7f6974013000 comp rx=0 tx=0).stop 2023-06-08T03:51:21.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.653+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 msgr2=0x7f6984157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.653+0000 7f6989832700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 0x7f6984157e50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f696c011b00 tx=0x7f696c011ec0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 shutdown_connections 2023-06-08T03:51:21.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f69840c7e00 0x7f69840c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f696406ace0 0x7f696406d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6970003a00 0x7f69841564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 --2- 172.21.15.79:0/1231540493 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f69841579e0 0x7f6984157e50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 >> 172.21.15.79:0/1231540493 conn(0x7f69840ce4e0 msgr2=0x7f69840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:21.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 shutdown_connections 2023-06-08T03:51:21.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.654+0000 7f6989832700 1 -- 172.21.15.79:0/1231540493 wait complete. 2023-06-08T03:51:21.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:21.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.788+0000 7f312a309700 1 Processor -- start 2023-06-08T03:51:21.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.788+0000 7f312a309700 1 -- start start 2023-06-08T03:51:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.788+0000 7f312a309700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f312415c1f0 0x7f312406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.788+0000 7f312a309700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3124069d50 0x7f312406a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.788+0000 7f312a309700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31240cd140 con 0x7f3124071fe0 2023-06-08T03:51:21.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f312a309700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312406a710 con 0x7f3124069d50 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f312a309700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3124071110 con 0x7f312415c1f0 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3124069d50 0x7f312406a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3124069d50 0x7f312406a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36508/0 (socket says 172.21.15.79:36508) 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 -- 172.21.15.79:0/365370370 learned_addr learned my addr 172.21.15.79:0/365370370 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f3123fff700 1 --2- 172.21.15.79:0/365370370 >> v2:172.21.15.156:3300/0 conn(0x7f312415c1f0 0x7f312406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 -- 172.21.15.79:0/365370370 >> v2:172.21.15.156:3300/0 conn(0x7f312415c1f0 msgr2=0x7f312406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 --2- 172.21.15.79:0/365370370 >> v2:172.21.15.156:3300/0 conn(0x7f312415c1f0 0x7f312406d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 -- 172.21.15.79:0/365370370 >> v1:172.21.15.156:6789/0 conn(0x7f3124071fe0 legacy=0x7f31240cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 -- 172.21.15.79:0/365370370 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124071be0 con 0x7f3124069d50 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f3123fff700 1 --2- 172.21.15.79:0/365370370 >> v2:172.21.15.156:3300/0 conn(0x7f312415c1f0 0x7f312406d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f31288a6700 1 --2- 172.21.15.79:0/365370370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124069d50 0x7f312406a1d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3124070d60 tx=0x7f311c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19367d102bbff3c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f3122ffd700 1 -- 172.21.15.79:0/365370370 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00e070 con 0x7f3124069d50 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f3122ffd700 1 -- 172.21.15.79:0/365370370 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c003a20 con 0x7f3124069d50 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.789+0000 7f3122ffd700 1 -- 172.21.15.79:0/365370370 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c003dd0 con 0x7f3124069d50 2023-06-08T03:51:21.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 -- 172.21.15.79:0/365370370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124069d50 msgr2=0x7f312406a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 --2- 172.21.15.79:0/365370370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124069d50 0x7f312406a1d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3124070d60 tx=0x7f311c00b330 comp rx=0 tx=0).stop 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 -- 172.21.15.79:0/365370370 shutdown_connections 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 --2- 172.21.15.79:0/365370370 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124069d50 0x7f312406a1d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 --2- 172.21.15.79:0/365370370 >> v2:172.21.15.156:3300/0 conn(0x7f312415c1f0 0x7f312406d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 -- 172.21.15.79:0/365370370 >> 172.21.15.79:0/365370370 conn(0x7f31240b1d30 msgr2=0x7f312406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 -- 172.21.15.79:0/365370370 shutdown_connections 2023-06-08T03:51:21.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.790+0000 7f312a309700 1 -- 172.21.15.79:0/365370370 wait complete. 2023-06-08T03:51:21.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.791+0000 7f312a309700 1 Processor -- start 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.791+0000 7f312a309700 1 -- start start 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f312a309700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3124069d50 0x7f312415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f312a309700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 0x7f31240c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f312a309700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f312415c1f0 0x7f3124150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f312a309700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312415ef40 con 0x7f3124071fe0 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f312a309700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31240c3d80 con 0x7f3124069d50 2023-06-08T03:51:21.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f312a309700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31240c4060 con 0x7f312415c1f0 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 0x7f31240c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 0x7f31240c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36512/0 (socket says 172.21.15.79:36512) 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 -- 172.21.15.79:0/2414619605 learned_addr learned my addr 172.21.15.79:0/2414619605 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31288a6700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f312415c1f0 0x7f3124150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f3123fff700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3124069d50 0x7f312415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 -- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f312415c1f0 msgr2=0x7f3124150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f312415c1f0 0x7f3124150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 -- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3124069d50 msgr2=0x7f312415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3124069d50 0x7f312415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 -- 172.21.15.79:0/2414619605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c009b30 con 0x7f3124071fe0 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f3123fff700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3124069d50 0x7f312415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.792+0000 7f31237fe700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 0x7f31240c58b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f311800b4a0 tx=0x7f311800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.793+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011830 con 0x7f3124071fe0 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.793+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118011990 con 0x7f3124071fe0 2023-06-08T03:51:21.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.793+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31240c4340 con 0x7f3124071fe0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.793+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118011d40 con 0x7f3124071fe0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.793+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31241f1ad0 con 0x7f3124071fe0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.793+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3124073e90 con 0x7f3124071fe0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.794+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f311801a400 con 0x7f3124071fe0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.794+0000 7f3120ff9700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f311006ace0 0x7f311006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.795+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3118089330 con 0x7f3124071fe0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.795+0000 7f3123fff700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f311006ace0 0x7f311006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:21.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.795+0000 7f3123fff700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f311006ace0 0x7f311006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f311400aa20 tx=0x7f3114008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:21.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.799+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3118054180 con 0x7f3124071fe0 2023-06-08T03:51:21.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.985+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f312406ed70 con 0x7f3124071fe0 2023-06-08T03:51:21.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.986+0000 7f3120ff9700 1 -- 172.21.15.79:0/2414619605 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31180577a0 con 0x7f3124071fe0 2023-06-08T03:51:21.990 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.987+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f311006ace0 msgr2=0x7f311006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.987+0000 7f312a309700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f311006ace0 0x7f311006d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f311400aa20 tx=0x7f3114008040 comp rx=0 tx=0).stop 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.987+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 msgr2=0x7f31240c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.987+0000 7f312a309700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 0x7f31240c58b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f311800b4a0 tx=0x7f311800b7b0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 shutdown_connections 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f311006ace0 0x7f311006d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f312415c1f0 0x7f3124150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3124069d50 0x7f312415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 --2- 172.21.15.79:0/2414619605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3124071fe0 0x7f31240c58b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:21.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 >> 172.21.15.79:0/2414619605 conn(0x7f31240b1d30 msgr2=0x7f31240c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:21.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 shutdown_connections 2023-06-08T03:51:21.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:21.988+0000 7f312a309700 1 -- 172.21.15.79:0/2414619605 wait complete. 2023-06-08T03:51:22.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:22.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.122+0000 7f19e15ea700 1 Processor -- start 2023-06-08T03:51:22.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.122+0000 7f19e15ea700 1 -- start start 2023-06-08T03:51:22.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19e15ea700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc15c1f0 0x7f19dc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19e15ea700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19dc069d50 0x7f19dc06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19e15ea700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc0cd140 con 0x7f19dc071fe0 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19e15ea700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc06a710 con 0x7f19dc069d50 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19e15ea700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc071110 con 0x7f19dc15c1f0 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19db7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19dc069d50 0x7f19dc06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19da7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f19dc071fe0 0x7f19dc0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59996/0 (socket says 172.21.15.79:59996) 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19da7fc700 1 -- 172.21.15.79:0/2443554001 learned_addr learned my addr 172.21.15.79:0/2443554001 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.123+0000 7f19daffd700 1 --2- 172.21.15.79:0/2443554001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc15c1f0 0x7f19dc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.124+0000 7f19db7fe700 1 -- 172.21.15.79:0/2443554001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc15c1f0 msgr2=0x7f19dc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.124+0000 7f19db7fe700 1 --2- 172.21.15.79:0/2443554001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc15c1f0 0x7f19dc06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.124+0000 7f19db7fe700 1 -- 172.21.15.79:0/2443554001 >> v1:172.21.15.79:6789/0 conn(0x7f19dc071fe0 legacy=0x7f19dc0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.124+0000 7f19d9ffb700 1 -- 172.21.15.79:0/2443554001 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834415064 0 0) 0x7f19dc0cd140 con 0x7f19dc071fe0 2023-06-08T03:51:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.124+0000 7f19db7fe700 1 -- 172.21.15.79:0/2443554001 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc071be0 con 0x7f19dc069d50 2023-06-08T03:51:22.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.124+0000 7f19db7fe700 1 --2- 172.21.15.79:0/2443554001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc069d50 0x7f19dc06a1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009d90 tx=0x7f19d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ee247a41829486 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.128+0000 7f19d9ffb700 1 -- 172.21.15.79:0/2443554001 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400e070 con 0x7f19dc069d50 2023-06-08T03:51:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.128+0000 7f19d9ffb700 1 -- 172.21.15.79:0/2443554001 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d400bb20 con 0x7f19dc069d50 2023-06-08T03:51:22.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.128+0000 7f19d9ffb700 1 -- 172.21.15.79:0/2443554001 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4012430 con 0x7f19dc069d50 2023-06-08T03:51:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 -- 172.21.15.79:0/2443554001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc069d50 msgr2=0x7f19dc06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2443554001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc069d50 0x7f19dc06a1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009d90 tx=0x7f19d400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 -- 172.21.15.79:0/2443554001 shutdown_connections 2023-06-08T03:51:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2443554001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc069d50 0x7f19dc06a1d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2443554001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc15c1f0 0x7f19dc06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 -- 172.21.15.79:0/2443554001 >> 172.21.15.79:0/2443554001 conn(0x7f19dc0b1d30 msgr2=0x7f19dc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 -- 172.21.15.79:0/2443554001 shutdown_connections 2023-06-08T03:51:22.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.129+0000 7f19e15ea700 1 -- 172.21.15.79:0/2443554001 wait complete. 2023-06-08T03:51:22.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.130+0000 7f19e15ea700 1 Processor -- start 2023-06-08T03:51:22.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 -- start start 2023-06-08T03:51:22.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc069d50 0x7f19dc0c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19dc071fe0 0x7f19dc15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 0x7f19dc159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc0c6640 con 0x7f19dc15c1f0 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc15d060 con 0x7f19dc071fe0 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19e15ea700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc15d340 con 0x7f19dc069d50 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19db7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 0x7f19dc159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19db7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 0x7f19dc159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36528/0 (socket says 172.21.15.79:36528) 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19db7fe700 1 -- 172.21.15.79:0/2971300388 learned_addr learned my addr 172.21.15.79:0/2971300388 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:22.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19daffd700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc069d50 0x7f19dc0c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.131+0000 7f19da7fc700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19dc071fe0 0x7f19dc15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.132+0000 7f19db7fe700 1 -- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc069d50 msgr2=0x7f19dc0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.132+0000 7f19db7fe700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc069d50 0x7f19dc0c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.132+0000 7f19db7fe700 1 -- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19dc071fe0 msgr2=0x7f19dc15c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.132+0000 7f19db7fe700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19dc071fe0 0x7f19dc15c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.132+0000 7f19db7fe700 1 -- 172.21.15.79:0/2971300388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009a70 con 0x7f19dc15c1f0 2023-06-08T03:51:22.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.132+0000 7f19db7fe700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 0x7f19dc159fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f19dc070770 tx=0x7f19d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.133+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4012430 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.133+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4020070 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.133+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc0c6850 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.133+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4004280 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.133+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc15adb0 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.134+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc0c6d70 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.134+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f19d40043e0 con 0x7f19dc15c1f0 2023-06-08T03:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.135+0000 7f19c37fe700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19c806ace0 0x7f19c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.135+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f19d4022070 con 0x7f19dc15c1f0 2023-06-08T03:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.135+0000 7f19daffd700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19c806ace0 0x7f19c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.135+0000 7f19daffd700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19c806ace0 0x7f19c806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009b40 tx=0x7f19d0009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.143 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.139+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f19dc0c6d70 con 0x7f19dc15c1f0 2023-06-08T03:51:22.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.324+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19dc0c6d70 con 0x7f19dc15c1f0 2023-06-08T03:51:22.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.324+0000 7f19c37fe700 1 -- 172.21.15.79:0/2971300388 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19dc0c6d70 con 0x7f19dc15c1f0 2023-06-08T03:51:22.328 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19c806ace0 msgr2=0x7f19c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19c806ace0 0x7f19c806d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009b40 tx=0x7f19d0009210 comp rx=0 tx=0).stop 2023-06-08T03:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 msgr2=0x7f19dc159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 0x7f19dc159fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f19dc070770 tx=0x7f19d4004ad0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 shutdown_connections 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19dc15c1f0 0x7f19dc159fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19c806ace0 0x7f19c806d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19dc069d50 0x7f19dc0c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 --2- 172.21.15.79:0/2971300388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19dc071fe0 0x7f19dc15c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 >> 172.21.15.79:0/2971300388 conn(0x7f19dc0b1d30 msgr2=0x7f19dc0b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:22.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 shutdown_connections 2023-06-08T03:51:22.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.326+0000 7f19e15ea700 1 -- 172.21.15.79:0/2971300388 wait complete. 2023-06-08T03:51:22.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:22.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.460+0000 7f5757118700 1 Processor -- start 2023-06-08T03:51:22.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.461+0000 7f5757118700 1 -- start start 2023-06-08T03:51:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.461+0000 7f5757118700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f57500cd080 0x7f57500cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.461+0000 7f5757118700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f57500ce480 0x7f57500ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.461+0000 7f5757118700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57500cae50 con 0x7f57500d0060 2023-06-08T03:51:22.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.461+0000 7f5757118700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57500cbea0 con 0x7f57500cd080 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.461+0000 7f5757118700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57500cda30 con 0x7f57500ce480 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574ffff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f57500cd080 0x7f57500cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f5754eb4700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f57500d0060 0x7f57500af7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60008/0 (socket says 172.21.15.79:60008) 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f5754eb4700 1 -- 172.21.15.79:0/3853412084 learned_addr learned my addr 172.21.15.79:0/3853412084 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f57556b5700 1 --2- 172.21.15.79:0/3853412084 >> v2:172.21.15.156:3300/0 conn(0x7f57500ce480 0x7f57500ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574ffff700 1 -- 172.21.15.79:0/3853412084 >> v2:172.21.15.156:3300/0 conn(0x7f57500ce480 msgr2=0x7f57500ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574ffff700 1 --2- 172.21.15.79:0/3853412084 >> v2:172.21.15.156:3300/0 conn(0x7f57500ce480 0x7f57500ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574ffff700 1 -- 172.21.15.79:0/3853412084 >> v1:172.21.15.79:6789/0 conn(0x7f57500d0060 legacy=0x7f57500af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574ffff700 1 -- 172.21.15.79:0/3853412084 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500afee0 con 0x7f57500cd080 2023-06-08T03:51:22.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574f7fe700 1 -- 172.21.15.79:0/3853412084 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251537673 0 0) 0x7f57500cae50 con 0x7f57500d0060 2023-06-08T03:51:22.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574ffff700 1 --2- 172.21.15.79:0/3853412084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500cd080 0x7f57500cd4f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f57500cc130 tx=0x7f573c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71ec7acaf170ea1 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574f7fe700 1 -- 172.21.15.79:0/3853412084 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c011830 con 0x7f57500cd080 2023-06-08T03:51:22.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574f7fe700 1 -- 172.21.15.79:0/3853412084 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c011990 con 0x7f57500cd080 2023-06-08T03:51:22.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.462+0000 7f574f7fe700 1 -- 172.21.15.79:0/3853412084 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c011d20 con 0x7f57500cd080 2023-06-08T03:51:22.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 -- 172.21.15.79:0/3853412084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500cd080 msgr2=0x7f57500cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 --2- 172.21.15.79:0/3853412084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500cd080 0x7f57500cd4f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f57500cc130 tx=0x7f573c009b20 comp rx=0 tx=0).stop 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 -- 172.21.15.79:0/3853412084 shutdown_connections 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 --2- 172.21.15.79:0/3853412084 >> v2:172.21.15.156:3300/0 conn(0x7f57500ce480 0x7f57500ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 --2- 172.21.15.79:0/3853412084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500cd080 0x7f57500cd4f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 -- 172.21.15.79:0/3853412084 >> 172.21.15.79:0/3853412084 conn(0x7f575006e1e0 msgr2=0x7f575006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 -- 172.21.15.79:0/3853412084 shutdown_connections 2023-06-08T03:51:22.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.463+0000 7f5757118700 1 -- 172.21.15.79:0/3853412084 wait complete. 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.464+0000 7f5757118700 1 Processor -- start 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.464+0000 7f5757118700 1 -- start start 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57500cd080 0x7f57500c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57500ce480 0x7f57500c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 0x7f57500c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57501f5fc0 con 0x7f57500d0060 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57501f50f0 con 0x7f57500cd080 2023-06-08T03:51:22.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57501f5340 con 0x7f57500ce480 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 0x7f57500c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 0x7f57500c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36542/0 (socket says 172.21.15.79:36542) 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 -- 172.21.15.79:0/2558446077 learned_addr learned my addr 172.21.15.79:0/2558446077 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f574ffff700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57500ce480 0x7f57500c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5754eb4700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57500cd080 0x7f57500c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 -- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57500ce480 msgr2=0x7f57500c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57500ce480 0x7f57500c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 -- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57500cd080 msgr2=0x7f57500c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57500cd080 0x7f57500c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 -- 172.21.15.79:0/2558446077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0094a0 con 0x7f57500d0060 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f57556b5700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 0x7f57500c4de0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f574800cc30 tx=0x7f574800a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574800ab70 con 0x7f57500d0060 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574800acd0 con 0x7f57500d0060 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57501f5620 con 0x7f57500d0060 2023-06-08T03:51:22.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.465+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748003c40 con 0x7f57500d0060 2023-06-08T03:51:22.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.466+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57501f5be0 con 0x7f57500d0060 2023-06-08T03:51:22.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.466+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5750073e90 con 0x7f57500d0060 2023-06-08T03:51:22.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.467+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f574800f3f0 con 0x7f57500d0060 2023-06-08T03:51:22.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.467+0000 7f574d7fa700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f573406ace0 0x7f573406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.467+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f57480898c0 con 0x7f57500d0060 2023-06-08T03:51:22.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.467+0000 7f5754eb4700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f573406ace0 0x7f573406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.468+0000 7f5754eb4700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f573406ace0 0x7f573406d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5744000f60 tx=0x7f5744007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.471+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5748054710 con 0x7f57500d0060 2023-06-08T03:51:22.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.657+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57500afcc0 con 0x7f57500d0060 2023-06-08T03:51:22.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.657+0000 7f574d7fa700 1 -- 172.21.15.79:0/2558446077 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f574801d0a0 con 0x7f57500d0060 2023-06-08T03:51:22.662 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:22.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f573406ace0 msgr2=0x7f573406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f573406ace0 0x7f573406d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5744000f60 tx=0x7f5744007c90 comp rx=0 tx=0).stop 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 msgr2=0x7f57500c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 0x7f57500c4de0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f574800cc30 tx=0x7f574800a1f0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 shutdown_connections 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57500d0060 0x7f57500c4de0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57500cd080 0x7f57500c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f573406ace0 0x7f573406d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 --2- 172.21.15.79:0/2558446077 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57500ce480 0x7f57500c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 >> 172.21.15.79:0/2558446077 conn(0x7f575006e1e0 msgr2=0x7f575006c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:22.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 shutdown_connections 2023-06-08T03:51:22.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.659+0000 7f5757118700 1 -- 172.21.15.79:0/2558446077 wait complete. 2023-06-08T03:51:22.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.793+0000 7f7d6de19700 1 Processor -- start 2023-06-08T03:51:22.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.793+0000 7f7d6de19700 1 -- start start 2023-06-08T03:51:22.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d6de19700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d68158d50 0x7f7d680ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d6de19700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7d6815d3d0 0x7f7d680cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d6de19700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d680cc920 con 0x7f7d680cc4c0 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d6de19700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d680cd7b0 con 0x7f7d6815d3d0 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d6de19700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680cda50 con 0x7f7d68158d50 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d66ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7d6815d3d0 0x7f7d680cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d677fe700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d68158d50 0x7f7d680ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d66ffd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7d6815d3d0 0x7f7d680cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:59440/0 (socket says 172.21.15.79:59440) 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d67fff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7d680cc4c0 0x7f7d6806eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:33954/0 (socket says 172.21.15.79:33954) 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.794+0000 7f7d66ffd700 1 -- 172.21.15.79:0/317886518 learned_addr learned my addr 172.21.15.79:0/317886518 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d66ffd700 1 -- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d68158d50 msgr2=0x7f7d680ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d66ffd700 1 --2- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d68158d50 0x7f7d680ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d66ffd700 1 -- 172.21.15.79:0/317886518 >> v1:172.21.15.156:6789/0 conn(0x7f7d680cc4c0 legacy=0x7f7d6806eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d66ffd700 1 -- 172.21.15.79:0/317886518 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680cdd30 con 0x7f7d6815d3d0 2023-06-08T03:51:22.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d667fc700 1 -- 172.21.15.79:0/317886518 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629255568 0 0) 0x7f7d680cc920 con 0x7f7d680cc4c0 2023-06-08T03:51:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d677fe700 1 --2- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d68158d50 0x7f7d680ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:22.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d66ffd700 1 --2- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d6815d3d0 0x7f7d680cbf80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7d580098f0 tx=0x7f7d58009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6212e0ca827e46e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d667fc700 1 -- 172.21.15.79:0/317886518 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58011830 con 0x7f7d6815d3d0 2023-06-08T03:51:22.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d667fc700 1 -- 172.21.15.79:0/317886518 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58011990 con 0x7f7d6815d3d0 2023-06-08T03:51:22.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.795+0000 7f7d667fc700 1 -- 172.21.15.79:0/317886518 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58011d20 con 0x7f7d6815d3d0 2023-06-08T03:51:22.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 -- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d6815d3d0 msgr2=0x7f7d680cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 --2- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d6815d3d0 0x7f7d680cbf80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7d580098f0 tx=0x7f7d58009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:22.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 -- 172.21.15.79:0/317886518 shutdown_connections 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 --2- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d68158d50 0x7f7d680ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 --2- 172.21.15.79:0/317886518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d6815d3d0 0x7f7d680cbf80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 -- 172.21.15.79:0/317886518 >> 172.21.15.79:0/317886518 conn(0x7f7d680b1d30 msgr2=0x7f7d6806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 -- 172.21.15.79:0/317886518 shutdown_connections 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.796+0000 7f7d6de19700 1 -- 172.21.15.79:0/317886518 wait complete. 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.797+0000 7f7d6de19700 1 Processor -- start 2023-06-08T03:51:22.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.797+0000 7f7d6de19700 1 -- start start 2023-06-08T03:51:22.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d6de19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d680cc4c0 0x7f7d680c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d6de19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 0x7f7d680c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d6de19700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d6815d3d0 0x7f7d680b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d6de19700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680c4720 con 0x7f7d68158d50 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d6de19700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680c4950 con 0x7f7d680cc4c0 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d6de19700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d680c4c30 con 0x7f7d6815d3d0 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 0x7f7d680c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 0x7f7d680c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36552/0 (socket says 172.21.15.79:36552) 2023-06-08T03:51:22.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.798+0000 7f7d66ffd700 1 -- 172.21.15.79:0/1455481599 learned_addr learned my addr 172.21.15.79:0/1455481599 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d67fff700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d6815d3d0 0x7f7d680b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d677fe700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d680cc4c0 0x7f7d680c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d66ffd700 1 -- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d6815d3d0 msgr2=0x7f7d680b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d66ffd700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d6815d3d0 0x7f7d680b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d66ffd700 1 -- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d680cc4c0 msgr2=0x7f7d680c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d66ffd700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d680cc4c0 0x7f7d680c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d66ffd700 1 -- 172.21.15.79:0/1455481599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5000b740 con 0x7f7d68158d50 2023-06-08T03:51:22.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d66ffd700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 0x7f7d680c41e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7d58014040 tx=0x7f7d58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004380 con 0x7f7d68158d50 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5801f070 con 0x7f7d68158d50 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58009550 con 0x7f7d68158d50 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580036a0 con 0x7f7d68158d50 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.799+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d680c51f0 con 0x7f7d68158d50 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.800+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68070870 con 0x7f7d68158d50 2023-06-08T03:51:22.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.800+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d58021030 con 0x7f7d68158d50 2023-06-08T03:51:22.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.801+0000 7f7d5ffff700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7d4806ace0 0x7f7d4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:22.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.801+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d5808ab70 con 0x7f7d68158d50 2023-06-08T03:51:22.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.801+0000 7f7d677fe700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7d4806ace0 0x7f7d4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:22.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.801+0000 7f7d677fe700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7d4806ace0 0x7f7d4806d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d680c38b0 tx=0x7f7d5000b420 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:22.809 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.805+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d58055ae0 con 0x7f7d68158d50 2023-06-08T03:51:22.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.989+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7d68073e90 con 0x7f7d68158d50 2023-06-08T03:51:22.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.990+0000 7f7d5ffff700 1 -- 172.21.15.79:0/1455481599 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7d58059100 con 0x7f7d68158d50 2023-06-08T03:51:22.994 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:22.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7d4806ace0 msgr2=0x7f7d4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7d4806ace0 0x7f7d4806d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d680c38b0 tx=0x7f7d5000b420 comp rx=0 tx=0).stop 2023-06-08T03:51:22.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 msgr2=0x7f7d680c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:22.996 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 0x7f7d680c41e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7d58014040 tx=0x7f7d58004750 comp rx=0 tx=0).stop 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 shutdown_connections 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7d4806ace0 0x7f7d4806d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d6815d3d0 0x7f7d680b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d680cc4c0 0x7f7d680c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 --2- 172.21.15.79:0/1455481599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7d68158d50 0x7f7d680c41e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 >> 172.21.15.79:0/1455481599 conn(0x7f7d680b1d30 msgr2=0x7f7d681558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 shutdown_connections 2023-06-08T03:51:22.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:22.992+0000 7f7d6de19700 1 -- 172.21.15.79:0/1455481599 wait complete. 2023-06-08T03:51:23.008 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.124+0000 7f8dc5cea700 1 Processor -- start 2023-06-08T03:51:23.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.124+0000 7f8dc5cea700 1 -- start start 2023-06-08T03:51:23.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dc5cea700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8dc015c1f0 0x7f8dc006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dc5cea700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8dc0071fe0 0x7f8dc00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dc5cea700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc00cf130 0x7f8dc0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dc5cea700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc00cd140 con 0x7f8dc015c1f0 2023-06-08T03:51:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dc5cea700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc0071110 con 0x7f8dc0071fe0 2023-06-08T03:51:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dc5cea700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc0071be0 con 0x7f8dc00cf130 2023-06-08T03:51:23.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8dc015c1f0 0x7f8dc006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8dc015c1f0 0x7f8dc006d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36558/0 (socket says 172.21.15.79:36558) 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 -- 172.21.15.79:0/1579072817 learned_addr learned my addr 172.21.15.79:0/1579072817 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbffff700 1 --2- 172.21.15.79:0/1579072817 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc00cf130 0x7f8dc0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbeffd700 1 --2- 172.21.15.79:0/1579072817 >> v2:172.21.15.156:3300/0 conn(0x7f8dc0071fe0 0x7f8dc00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 -- 172.21.15.79:0/1579072817 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc00cf130 msgr2=0x7f8dc0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 --2- 172.21.15.79:0/1579072817 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc00cf130 0x7f8dc0069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 -- 172.21.15.79:0/1579072817 >> v2:172.21.15.156:3300/0 conn(0x7f8dc0071fe0 msgr2=0x7f8dc00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:23.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 --2- 172.21.15.79:0/1579072817 >> v2:172.21.15.156:3300/0 conn(0x7f8dc0071fe0 0x7f8dc00cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbf7fe700 1 -- 172.21.15.79:0/1579072817 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00cd390 con 0x7f8dc015c1f0 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.125+0000 7f8dbeffd700 1 --2- 172.21.15.79:0/1579072817 >> v2:172.21.15.156:3300/0 conn(0x7f8dc0071fe0 0x7f8dc00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dbf7fe700 1 --2- 172.21.15.79:0/1579072817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc006d8a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00d0670 tx=0x7f8da8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd88838ce4de1ed server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dbe7fc700 1 -- 172.21.15.79:0/1579072817 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800b870 con 0x7f8dc015c1f0 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dbe7fc700 1 -- 172.21.15.79:0/1579072817 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800b9d0 con 0x7f8dc015c1f0 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dbe7fc700 1 -- 172.21.15.79:0/1579072817 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da800bd60 con 0x7f8dc015c1f0 2023-06-08T03:51:23.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 -- 172.21.15.79:0/1579072817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 msgr2=0x7f8dc006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/1579072817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc006d8a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00d0670 tx=0x7f8da8009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 -- 172.21.15.79:0/1579072817 shutdown_connections 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/1579072817 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc00cf130 0x7f8dc0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/1579072817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc006d8a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/1579072817 >> v2:172.21.15.156:3300/0 conn(0x7f8dc0071fe0 0x7f8dc00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 -- 172.21.15.79:0/1579072817 >> 172.21.15.79:0/1579072817 conn(0x7f8dc00b1d30 msgr2=0x7f8dc006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 -- 172.21.15.79:0/1579072817 shutdown_connections 2023-06-08T03:51:23.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.126+0000 7f8dc5cea700 1 -- 172.21.15.79:0/1579072817 wait complete. 2023-06-08T03:51:23.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.128+0000 7f8dc5cea700 1 Processor -- start 2023-06-08T03:51:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.128+0000 7f8dc5cea700 1 -- start start 2023-06-08T03:51:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dc5cea700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc0071fe0 0x7f8dc00b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dc5cea700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc00cf130 0x7f8dc01526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dc5cea700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dc5cea700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00c99e0 con 0x7f8dc015c1f0 2023-06-08T03:51:23.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dc5cea700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00b3590 con 0x7f8dc00cf130 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dc5cea700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc00b3870 con 0x7f8dc0071fe0 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dbffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dbffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc0152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36568/0 (socket says 172.21.15.79:36568) 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dbffff700 1 -- 172.21.15.79:0/2940906895 learned_addr learned my addr 172.21.15.79:0/2940906895 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dbf7fe700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc0071fe0 0x7f8dc00b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.129+0000 7f8dbeffd700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc00cf130 0x7f8dc01526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dbffff700 1 -- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc0071fe0 msgr2=0x7f8dc00b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dbffff700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc0071fe0 0x7f8dc00b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dbffff700 1 -- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc00cf130 msgr2=0x7f8dc01526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dbffff700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc00cf130 0x7f8dc01526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dbffff700 1 -- 172.21.15.79:0/2940906895 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da8009550 con 0x7f8dc015c1f0 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dbffff700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc0152be0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8db8007e40 tx=0x7f8db800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800ed30 con 0x7f8dc015c1f0 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db8014420 con 0x7f8dc015c1f0 2023-06-08T03:51:23.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc00b3b50 con 0x7f8dc015c1f0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db80147d0 con 0x7f8dc015c1f0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.130+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00b4110 con 0x7f8dc015c1f0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.131+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc0073e90 con 0x7f8dc015c1f0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.131+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8db8014930 con 0x7f8dc015c1f0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.132+0000 7f8db7fff700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8da006ace0 0x7f8da006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.132+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8db8089e30 con 0x7f8dc015c1f0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.132+0000 7f8dbf7fe700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8da006ace0 0x7f8da006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.140 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.132+0000 7f8dbf7fe700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8da006ace0 0x7f8da006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da8008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:23.141 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.136+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8db8054bf0 con 0x7f8dc015c1f0 2023-06-08T03:51:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.338+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8dc006ed70 con 0x7f8dc015c1f0 2023-06-08T03:51:23.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.338+0000 7f8db7fff700 1 -- 172.21.15.79:0/2940906895 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8db8058210 con 0x7f8dc015c1f0 2023-06-08T03:51:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8da006ace0 msgr2=0x7f8da006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8da006ace0 0x7f8da006d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da8008030 comp rx=0 tx=0).stop 2023-06-08T03:51:23.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 msgr2=0x7f8dc0152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc0152be0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8db8007e40 tx=0x7f8db800e3c0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 shutdown_connections 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8dc015c1f0 0x7f8dc0152be0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8da006ace0 0x7f8da006d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8dc0071fe0 0x7f8dc00b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 --2- 172.21.15.79:0/2940906895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8dc00cf130 0x7f8dc01526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 >> 172.21.15.79:0/2940906895 conn(0x7f8dc00b1d30 msgr2=0x7f8dc00ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:23.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 shutdown_connections 2023-06-08T03:51:23.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.340+0000 7f8dc5cea700 1 -- 172.21.15.79:0/2940906895 wait complete. 2023-06-08T03:51:23.346 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:23.356 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi156.front.sepia.ceph.com... 2023-06-08T03:51:23.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:23.357 DEBUG:teuthology.orchestra.run.smithi156:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-06-08T03:51:23.402 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:23.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:51:23.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:23.403 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-08T03:51:23.432 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:51:23.432 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:23.450 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:51:23.450 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:23.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:23.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.661+0000 7fba75867700 1 Processor -- start 2023-06-08T03:51:23.665 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.661+0000 7fba75867700 1 -- start start 2023-06-08T03:51:23.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba75867700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba7006ae60 0x7fba7015c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba75867700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fba700b0010 0x7fba7006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba75867700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700b0460 con 0x7fba700ce2a0 2023-06-08T03:51:23.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba75867700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba700b12b0 con 0x7fba700b0010 2023-06-08T03:51:23.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba75867700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7006df50 con 0x7fba7006ae60 2023-06-08T03:51:23.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6e7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fba700b0010 0x7fba7006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6f7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fba700ce2a0 0x7fba700b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60052/0 (socket says 172.21.15.79:60052) 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6f7fe700 1 -- 172.21.15.79:0/3177488640 learned_addr learned my addr 172.21.15.79:0/3177488640 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6effd700 1 --2- 172.21.15.79:0/3177488640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba7006ae60 0x7fba7015c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6e7fc700 1 -- 172.21.15.79:0/3177488640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba7006ae60 msgr2=0x7fba7015c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6e7fc700 1 --2- 172.21.15.79:0/3177488640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba7006ae60 0x7fba7015c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6e7fc700 1 -- 172.21.15.79:0/3177488640 >> v1:172.21.15.79:6789/0 conn(0x7fba700ce2a0 legacy=0x7fba700b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6dffb700 1 -- 172.21.15.79:0/3177488640 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888679112 0 0) 0x7fba68001490 con 0x7fba700ce2a0 2023-06-08T03:51:23.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6e7fc700 1 -- 172.21.15.79:0/3177488640 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7006b770 con 0x7fba700b0010 2023-06-08T03:51:23.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.662+0000 7fba6e7fc700 1 --2- 172.21.15.79:0/3177488640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700b0010 0x7fba7006b230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fba60009840 tx=0x7fba60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ad83eff8afb221 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:23.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba6dffb700 1 -- 172.21.15.79:0/3177488640 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011830 con 0x7fba700b0010 2023-06-08T03:51:23.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba6dffb700 1 -- 172.21.15.79:0/3177488640 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba60011990 con 0x7fba700b0010 2023-06-08T03:51:23.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba6dffb700 1 -- 172.21.15.79:0/3177488640 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011d20 con 0x7fba700b0010 2023-06-08T03:51:23.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 -- 172.21.15.79:0/3177488640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700b0010 msgr2=0x7fba7006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 --2- 172.21.15.79:0/3177488640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700b0010 0x7fba7006b230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fba60009840 tx=0x7fba60009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 -- 172.21.15.79:0/3177488640 shutdown_connections 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 --2- 172.21.15.79:0/3177488640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba7006ae60 0x7fba7015c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 --2- 172.21.15.79:0/3177488640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700b0010 0x7fba7006b230 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 -- 172.21.15.79:0/3177488640 >> 172.21.15.79:0/3177488640 conn(0x7fba700999b0 msgr2=0x7fba700d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 -- 172.21.15.79:0/3177488640 shutdown_connections 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.663+0000 7fba75867700 1 -- 172.21.15.79:0/3177488640 wait complete. 2023-06-08T03:51:23.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.664+0000 7fba75867700 1 Processor -- start 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 -- start start 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba7006ae60 0x7fba701f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba700b0010 0x7fba700ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 0x7fba700aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba701f2c00 con 0x7fba700ce2a0 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba701f2120 con 0x7fba7006ae60 2023-06-08T03:51:23.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba75867700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba701f2f40 con 0x7fba700b0010 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba6f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 0x7fba700aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba6f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 0x7fba700aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36582/0 (socket says 172.21.15.79:36582) 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.665+0000 7fba6f7fe700 1 -- 172.21.15.79:0/663616878 learned_addr learned my addr 172.21.15.79:0/663616878 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6effd700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba7006ae60 0x7fba701f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6e7fc700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba700b0010 0x7fba700ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6f7fe700 1 -- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba700b0010 msgr2=0x7fba700ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6f7fe700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba700b0010 0x7fba700ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6f7fe700 1 -- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba7006ae60 msgr2=0x7fba701f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6f7fe700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba7006ae60 0x7fba701f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6f7fe700 1 -- 172.21.15.79:0/663616878 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600094a0 con 0x7fba700ce2a0 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6e7fc700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba700b0010 0x7fba700ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba6f7fe700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 0x7fba700aedf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba6800cca0 tx=0x7fba6800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6800cea0 con 0x7fba700ce2a0 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6800ada0 con 0x7fba700ce2a0 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba701f2360 con 0x7fba700ce2a0 2023-06-08T03:51:23.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68007310 con 0x7fba700ce2a0 2023-06-08T03:51:23.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.666+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba701f2920 con 0x7fba700ce2a0 2023-06-08T03:51:23.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.667+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba701f2da0 con 0x7fba700ce2a0 2023-06-08T03:51:23.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.667+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba68007470 con 0x7fba700ce2a0 2023-06-08T03:51:23.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.668+0000 7fba57fff700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fba5c06ace0 0x7fba5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:23.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.668+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fba680894e0 con 0x7fba700ce2a0 2023-06-08T03:51:23.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.668+0000 7fba6effd700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fba5c06ace0 0x7fba5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:23.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.669+0000 7fba6effd700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fba5c06ace0 0x7fba5c06d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fba6400b9b0 tx=0x7fba64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:23.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.672+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba701f2da0 con 0x7fba700ce2a0 2023-06-08T03:51:23.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.863+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba700b0460 con 0x7fba700ce2a0 2023-06-08T03:51:23.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.864+0000 7fba57fff700 1 -- 172.21.15.79:0/663616878 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fba700b0460 con 0x7fba700ce2a0 2023-06-08T03:51:23.868 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:23.868 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fba5c06ace0 msgr2=0x7fba5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fba5c06ace0 0x7fba5c06d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fba6400b9b0 tx=0x7fba64008040 comp rx=0 tx=0).stop 2023-06-08T03:51:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 msgr2=0x7fba700aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:23.870 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 0x7fba700aedf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fba6800cca0 tx=0x7fba6800a340 comp rx=0 tx=0).stop 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 shutdown_connections 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba700ce2a0 0x7fba700aedf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba7006ae60 0x7fba701f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fba5c06ace0 0x7fba5c06d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 --2- 172.21.15.79:0/663616878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba700b0010 0x7fba700ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 >> 172.21.15.79:0/663616878 conn(0x7fba700999b0 msgr2=0x7fba700cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:23.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 shutdown_connections 2023-06-08T03:51:23.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:23.866+0000 7fba75867700 1 -- 172.21.15.79:0/663616878 wait complete. 2023-06-08T03:51:23.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:23.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:24.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.001+0000 7efc27bda700 1 Processor -- start 2023-06-08T03:51:24.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.001+0000 7efc27bda700 1 -- start start 2023-06-08T03:51:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc27bda700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efc20158d50 0x7efc200ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc27bda700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc2015d3d0 0x7efc200cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc27bda700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200cc8c0 con 0x7efc200cc4c0 2023-06-08T03:51:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc27bda700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200cd7b0 con 0x7efc2015d3d0 2023-06-08T03:51:24.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc27bda700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200cda50 con 0x7efc20158d50 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc25175700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc2015d3d0 0x7efc200cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc25175700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efc2015d3d0 0x7efc200cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36588/0 (socket says 172.21.15.79:36588) 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.002+0000 7efc25175700 1 -- 172.21.15.79:0/766035698 learned_addr learned my addr 172.21.15.79:0/766035698 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25976700 1 --2- 172.21.15.79:0/766035698 >> v2:172.21.15.156:3300/0 conn(0x7efc20158d50 0x7efc200ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25175700 1 -- 172.21.15.79:0/766035698 >> v2:172.21.15.156:3300/0 conn(0x7efc20158d50 msgr2=0x7efc200ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25175700 1 --2- 172.21.15.79:0/766035698 >> v2:172.21.15.156:3300/0 conn(0x7efc20158d50 0x7efc200ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25175700 1 -- 172.21.15.79:0/766035698 >> v1:172.21.15.156:6789/0 conn(0x7efc200cc4c0 legacy=0x7efc2006eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:24.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25175700 1 -- 172.21.15.79:0/766035698 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200cdd30 con 0x7efc2015d3d0 2023-06-08T03:51:24.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25976700 1 --2- 172.21.15.79:0/766035698 >> v2:172.21.15.156:3300/0 conn(0x7efc20158d50 0x7efc200ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:24.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc25175700 1 --2- 172.21.15.79:0/766035698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc2015d3d0 0x7efc200cbf80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc2006e5d0 tx=0x7efc1c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c278de7310c87e36 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc24974700 1 -- 172.21.15.79:0/766035698 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00dcf0 con 0x7efc2015d3d0 2023-06-08T03:51:24.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc24974700 1 -- 172.21.15.79:0/766035698 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c00de50 con 0x7efc2015d3d0 2023-06-08T03:51:24.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc24974700 1 -- 172.21.15.79:0/766035698 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c004250 con 0x7efc2015d3d0 2023-06-08T03:51:24.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc27bda700 1 -- 172.21.15.79:0/766035698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc2015d3d0 msgr2=0x7efc200cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc27bda700 1 --2- 172.21.15.79:0/766035698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc2015d3d0 0x7efc200cbf80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efc2006e5d0 tx=0x7efc1c00be70 comp rx=0 tx=0).stop 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc27bda700 1 -- 172.21.15.79:0/766035698 shutdown_connections 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.003+0000 7efc27bda700 1 --2- 172.21.15.79:0/766035698 >> v2:172.21.15.156:3300/0 conn(0x7efc20158d50 0x7efc200ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.004+0000 7efc27bda700 1 --2- 172.21.15.79:0/766035698 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc2015d3d0 0x7efc200cbf80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.004+0000 7efc27bda700 1 -- 172.21.15.79:0/766035698 >> 172.21.15.79:0/766035698 conn(0x7efc200b1d30 msgr2=0x7efc2006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.004+0000 7efc27bda700 1 -- 172.21.15.79:0/766035698 shutdown_connections 2023-06-08T03:51:24.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.004+0000 7efc27bda700 1 -- 172.21.15.79:0/766035698 wait complete. 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.005+0000 7efc27bda700 1 Processor -- start 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.005+0000 7efc27bda700 1 -- start start 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.005+0000 7efc27bda700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 0x7efc20151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.005+0000 7efc27bda700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc20158d50 0x7efc200ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.005+0000 7efc27bda700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc2015d3d0 0x7efc200ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.005+0000 7efc27bda700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201f2090 con 0x7efc200cc4c0 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc27bda700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201f2330 con 0x7efc20158d50 2023-06-08T03:51:24.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc27bda700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc201f2610 con 0x7efc2015d3d0 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 0x7efc20151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 0x7efc20151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36592/0 (socket says 172.21.15.79:36592) 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 -- 172.21.15.79:0/1764614502 learned_addr learned my addr 172.21.15.79:0/1764614502 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25175700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc20158d50 0x7efc200ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc26177700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc2015d3d0 0x7efc200ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 -- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc2015d3d0 msgr2=0x7efc200ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc2015d3d0 0x7efc200ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 -- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc20158d50 msgr2=0x7efc200ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc20158d50 0x7efc200ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 -- 172.21.15.79:0/1764614502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c00b7f0 con 0x7efc200cc4c0 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc25976700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 0x7efc20151a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c750 tx=0x7efc1000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10011800 con 0x7efc200cc4c0 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10011960 con 0x7efc200cc4c0 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc201f28f0 con 0x7efc200cc4c0 2023-06-08T03:51:24.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10011cd0 con 0x7efc200cc4c0 2023-06-08T03:51:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.006+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc201f2e10 con 0x7efc200cc4c0 2023-06-08T03:51:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.007+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc201f2b10 con 0x7efc200cc4c0 2023-06-08T03:51:24.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.008+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc10021350 con 0x7efc200cc4c0 2023-06-08T03:51:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.008+0000 7efc167fc700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efc0406ace0 0x7efc0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.008+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efc10089750 con 0x7efc200cc4c0 2023-06-08T03:51:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.008+0000 7efc25175700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efc0406ace0 0x7efc0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.009+0000 7efc25175700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efc0406ace0 0x7efc0406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b340 tx=0x7efc1c012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.017 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.012+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efc10057fa0 con 0x7efc200cc4c0 2023-06-08T03:51:24.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.197+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efc20073e90 con 0x7efc200cc4c0 2023-06-08T03:51:24.202 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.198+0000 7efc167fc700 1 -- 172.21.15.79:0/1764614502 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7efc10057b30 con 0x7efc200cc4c0 2023-06-08T03:51:24.202 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:24.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efc0406ace0 msgr2=0x7efc0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efc0406ace0 0x7efc0406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b340 tx=0x7efc1c012000 comp rx=0 tx=0).stop 2023-06-08T03:51:24.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 msgr2=0x7efc20151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 0x7efc20151a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c750 tx=0x7efc1000cb10 comp rx=0 tx=0).stop 2023-06-08T03:51:24.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 shutdown_connections 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7efc0406ace0 0x7efc0406d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efc2015d3d0 0x7efc200ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc200cc4c0 0x7efc20151a10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 --2- 172.21.15.79:0/1764614502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc20158d50 0x7efc200ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 >> 172.21.15.79:0/1764614502 conn(0x7efc200b1d30 msgr2=0x7efc20156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 shutdown_connections 2023-06-08T03:51:24.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.200+0000 7efc27bda700 1 -- 172.21.15.79:0/1764614502 wait complete. 2023-06-08T03:51:24.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:24.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.334+0000 7f3505575700 1 Processor -- start 2023-06-08T03:51:24.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.334+0000 7f3505575700 1 -- start start 2023-06-08T03:51:24.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.334+0000 7f3505575700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f35000cd080 0x7f35000cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f3505575700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35000cae50 con 0x7f35000ce480 2023-06-08T03:51:24.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f3505575700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35000afee0 con 0x7f35000d0060 2023-06-08T03:51:24.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f3505575700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35000cbea0 con 0x7f35000cd080 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fe7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f35000cd080 0x7f35000cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34ff7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f35000ce480 0x7f35000ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60076/0 (socket says 172.21.15.79:60076) 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34ff7fe700 1 -- 172.21.15.79:0/4120663277 learned_addr learned my addr 172.21.15.79:0/4120663277 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fe7fc700 1 -- 172.21.15.79:0/4120663277 >> v1:172.21.15.156:6789/0 conn(0x7f35000d0060 legacy=0x7f35000af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fe7fc700 1 -- 172.21.15.79:0/4120663277 >> v1:172.21.15.79:6789/0 conn(0x7f35000ce480 legacy=0x7f35000ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fdffb700 1 -- 172.21.15.79:0/4120663277 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398639309 0 0) 0x7f35000cae50 con 0x7f35000ce480 2023-06-08T03:51:24.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fe7fc700 1 -- 172.21.15.79:0/4120663277 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350015d930 con 0x7f35000cd080 2023-06-08T03:51:24.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fe7fc700 1 --2- 172.21.15.79:0/4120663277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000cd4f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009840 tx=0x7f34f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b43624f1cff308 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fdffb700 1 -- 172.21.15.79:0/4120663277 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4004230 con 0x7f35000cd080 2023-06-08T03:51:24.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fdffb700 1 -- 172.21.15.79:0/4120663277 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f4004390 con 0x7f35000cd080 2023-06-08T03:51:24.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.335+0000 7f34fdffb700 1 -- 172.21.15.79:0/4120663277 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f40036a0 con 0x7f35000cd080 2023-06-08T03:51:24.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 -- 172.21.15.79:0/4120663277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 msgr2=0x7f35000cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 --2- 172.21.15.79:0/4120663277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000cd4f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009840 tx=0x7f34f4009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:24.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 -- 172.21.15.79:0/4120663277 shutdown_connections 2023-06-08T03:51:24.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 --2- 172.21.15.79:0/4120663277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000cd4f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 -- 172.21.15.79:0/4120663277 >> 172.21.15.79:0/4120663277 conn(0x7f350006e1e0 msgr2=0x7f350006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 -- 172.21.15.79:0/4120663277 shutdown_connections 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.336+0000 7f3505575700 1 -- 172.21.15.79:0/4120663277 wait complete. 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.337+0000 7f3505575700 1 Processor -- start 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.337+0000 7f3505575700 1 -- start start 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f3505575700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f3505575700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35000ce480 0x7f35001f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f3505575700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35000d0060 0x7f35000b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f3505575700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000b2170 con 0x7f35000cd080 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f3505575700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000b0b90 con 0x7f35000ce480 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f3505575700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35000b0e70 con 0x7f35000d0060 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36604/0 (socket says 172.21.15.79:36604) 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 -- 172.21.15.79:0/3738224906 learned_addr learned my addr 172.21.15.79:0/3738224906 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:24.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34ff7fe700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35000d0060 0x7f35000b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34fe7fc700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35000ce480 0x7f35001f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 -- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35000d0060 msgr2=0x7f35000b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35000d0060 0x7f35000b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 -- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35000ce480 msgr2=0x7f35001f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35000ce480 0x7f35001f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34feffd700 1 -- 172.21.15.79:0/3738224906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40094a0 con 0x7f35000cd080 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34ff7fe700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35000d0060 0x7f35000b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:24.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.338+0000 7f34fe7fc700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35000ce480 0x7f35001f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:24.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.339+0000 7f34feffd700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000b1c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c4d0 tx=0x7f34f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.339+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0011790 con 0x7f35000cd080 2023-06-08T03:51:24.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.339+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f00118f0 con 0x7f35000cd080 2023-06-08T03:51:24.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.339+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35000b1150 con 0x7f35000cd080 2023-06-08T03:51:24.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.339+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0011ca0 con 0x7f35000cd080 2023-06-08T03:51:24.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.339+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35000b1710 con 0x7f35000cd080 2023-06-08T03:51:24.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.340+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3500073e90 con 0x7f35000cd080 2023-06-08T03:51:24.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.340+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34f001c460 con 0x7f35000cd080 2023-06-08T03:51:24.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.340+0000 7f34e77fe700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f34ec06ace0 0x7f34ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.340+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f34f0089620 con 0x7f35000cd080 2023-06-08T03:51:24.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.341+0000 7f34fe7fc700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f34ec06ace0 0x7f34ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.341+0000 7f34fe7fc700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f34ec06ace0 0x7f34ec06d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009840 tx=0x7f34f4008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.345+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f34f00543e0 con 0x7f35000cd080 2023-06-08T03:51:24.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.529+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3500070120 con 0x7f35000cd080 2023-06-08T03:51:24.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.529+0000 7f34e77fe700 1 -- 172.21.15.79:0/3738224906 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34f0057a00 con 0x7f35000cd080 2023-06-08T03:51:24.533 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f34ec06ace0 msgr2=0x7f34ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f34ec06ace0 0x7f34ec06d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009840 tx=0x7f34f4008030 comp rx=0 tx=0).stop 2023-06-08T03:51:24.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 msgr2=0x7f35000b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000b1c30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c4d0 tx=0x7f34f000c890 comp rx=0 tx=0).stop 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 shutdown_connections 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f34ec06ace0 0x7f34ec06d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35000d0060 0x7f35000b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35000cd080 0x7f35000b1c30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 --2- 172.21.15.79:0/3738224906 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35000ce480 0x7f35001f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 >> 172.21.15.79:0/3738224906 conn(0x7f350006e1e0 msgr2=0x7f35000cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:24.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 shutdown_connections 2023-06-08T03:51:24.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.531+0000 7f3505575700 1 -- 172.21.15.79:0/3738224906 wait complete. 2023-06-08T03:51:24.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:24.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.665+0000 7fc2e0119700 1 Processor -- start 2023-06-08T03:51:24.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2e0119700 1 -- start start 2023-06-08T03:51:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2e0119700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2e0119700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2d806ae60 0x7fc2d806b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2e0119700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d815caa0 con 0x7fc2d806d410 2023-06-08T03:51:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2e0119700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d815cd80 con 0x7fc2d806ae60 2023-06-08T03:51:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2e0119700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d815d060 con 0x7fc2d806a860 2023-06-08T03:51:24.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.666+0000 7fc2de6b6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2d806ae60 0x7fc2d806b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc2d806ae60 0x7fc2d806b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36616/0 (socket says 172.21.15.79:36616) 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 -- 172.21.15.79:0/4226580786 learned_addr learned my addr 172.21.15.79:0/4226580786 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2dd6b4700 1 --2- 172.21.15.79:0/4226580786 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 -- 172.21.15.79:0/4226580786 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 msgr2=0x7fc2d806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 --2- 172.21.15.79:0/4226580786 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 -- 172.21.15.79:0/4226580786 >> v1:172.21.15.156:6789/0 conn(0x7fc2d806d410 legacy=0x7fc2d806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 -- 172.21.15.79:0/4226580786 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d815d340 con 0x7fc2d806ae60 2023-06-08T03:51:24.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2de6b6700 1 --2- 172.21.15.79:0/4226580786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d806b2e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009e50 tx=0x7fc2d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05c60c4c03f62d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2dceb3700 1 -- 172.21.15.79:0/4226580786 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000e070 con 0x7fc2d806ae60 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2dceb3700 1 -- 172.21.15.79:0/4226580786 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000b910 con 0x7fc2d806ae60 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.667+0000 7fc2dceb3700 1 -- 172.21.15.79:0/4226580786 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000bcc0 con 0x7fc2d806ae60 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 -- 172.21.15.79:0/4226580786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 msgr2=0x7fc2d806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 --2- 172.21.15.79:0/4226580786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d806b2e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009e50 tx=0x7fc2d000b330 comp rx=0 tx=0).stop 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 -- 172.21.15.79:0/4226580786 shutdown_connections 2023-06-08T03:51:24.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 --2- 172.21.15.79:0/4226580786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d806b2e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 --2- 172.21.15.79:0/4226580786 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 -- 172.21.15.79:0/4226580786 >> 172.21.15.79:0/4226580786 conn(0x7fc2d80ced30 msgr2=0x7fc2d80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:24.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 -- 172.21.15.79:0/4226580786 shutdown_connections 2023-06-08T03:51:24.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.668+0000 7fc2e0119700 1 -- 172.21.15.79:0/4226580786 wait complete. 2023-06-08T03:51:24.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.669+0000 7fc2e0119700 1 Processor -- start 2023-06-08T03:51:24.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.669+0000 7fc2e0119700 1 -- start start 2023-06-08T03:51:24.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d809e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d80c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d806d410 0x7fc2d809d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c5190 con 0x7fc2d806ae60 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d809ed50 con 0x7fc2d806d410 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d809e790 con 0x7fc2d806a860 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d80c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d80c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36620/0 (socket says 172.21.15.79:36620) 2023-06-08T03:51:24.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 -- 172.21.15.79:0/1168283098 learned_addr learned my addr 172.21.15.79:0/1168283098 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2ddeb5700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d809e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2de6b6700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d806d410 0x7fc2d809d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 -- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 msgr2=0x7fc2d809e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d809e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 -- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d806d410 msgr2=0x7fc2d809d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d806d410 0x7fc2d809d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 -- 172.21.15.79:0/1168283098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0009b30 con 0x7fc2d806ae60 2023-06-08T03:51:24.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2ddeb5700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d809e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2de6b6700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d806d410 0x7fc2d809d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2dd6b4700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d80c4c50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400e9b0 tx=0x7fc2d400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4009710 con 0x7fc2d806ae60 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d400c850 con 0x7fc2d806ae60 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d809ea70 con 0x7fc2d806ae60 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400cbc0 con 0x7fc2d806ae60 2023-06-08T03:51:24.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.670+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80b3030 con 0x7fc2d806ae60 2023-06-08T03:51:24.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.671+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d80b33a0 con 0x7fc2d806ae60 2023-06-08T03:51:24.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.672+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2d400cd20 con 0x7fc2d806ae60 2023-06-08T03:51:24.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.672+0000 7fc2ce7fc700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2bc06ace0 0x7fc2bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:24.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.672+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc2d4089870 con 0x7fc2d806ae60 2023-06-08T03:51:24.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.672+0000 7fc2ddeb5700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2bc06ace0 0x7fc2bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:24.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.673+0000 7fc2ddeb5700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2bc06ace0 0x7fc2bc06d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8000f60 tx=0x7fc2c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:24.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.676+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc2d40580c0 con 0x7fc2d806ae60 2023-06-08T03:51:24.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.861+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2d80b2bb0 con 0x7fc2d806ae60 2023-06-08T03:51:24.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.862+0000 7fc2ce7fc700 1 -- 172.21.15.79:0/1168283098 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2d4021020 con 0x7fc2d806ae60 2023-06-08T03:51:24.866 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:24.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2bc06ace0 msgr2=0x7fc2bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2bc06ace0 0x7fc2bc06d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8000f60 tx=0x7fc2c8009040 comp rx=0 tx=0).stop 2023-06-08T03:51:24.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 msgr2=0x7fc2d80c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:24.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d80c4c50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400e9b0 tx=0x7fc2d400ed70 comp rx=0 tx=0).stop 2023-06-08T03:51:24.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 shutdown_connections 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d806d410 0x7fc2d809d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc2bc06ace0 0x7fc2bc06d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d806a860 0x7fc2d809e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 --2- 172.21.15.79:0/1168283098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc2d806ae60 0x7fc2d80c4c50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 >> 172.21.15.79:0/1168283098 conn(0x7fc2d80ced30 msgr2=0x7fc2d815f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 shutdown_connections 2023-06-08T03:51:24.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:24.864+0000 7fc2e0119700 1 -- 172.21.15.79:0/1168283098 wait complete. 2023-06-08T03:51:24.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:25.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.000+0000 7f7e847d3700 1 Processor -- start 2023-06-08T03:51:25.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.000+0000 7f7e847d3700 1 -- start start 2023-06-08T03:51:25.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e847d3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7e7c15c1f0 0x7f7e7c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e847d3700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e7c069d50 0x7f7e7c06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e847d3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c0cd140 con 0x7f7e7c071fe0 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e847d3700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c06a710 con 0x7f7e7c069d50 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e847d3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c071110 con 0x7f7e7c15c1f0 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e82d70700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e7c069d50 0x7f7e7c06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e81d6e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7e7c071fe0 0x7f7e7c0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60102/0 (socket says 172.21.15.79:60102) 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.001+0000 7f7e81d6e700 1 -- 172.21.15.79:0/2073522477 learned_addr learned my addr 172.21.15.79:0/2073522477 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:25.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8256f700 1 --2- 172.21.15.79:0/2073522477 >> v2:172.21.15.156:3300/0 conn(0x7f7e7c15c1f0 0x7f7e7c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8156d700 1 -- 172.21.15.79:0/2073522477 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583853126 0 0) 0x7f7e7c0cd140 con 0x7f7e7c071fe0 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8156d700 1 -- 172.21.15.79:0/2073522477 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e680035b0 con 0x7f7e7c071fe0 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e82d70700 1 -- 172.21.15.79:0/2073522477 >> v2:172.21.15.156:3300/0 conn(0x7f7e7c15c1f0 msgr2=0x7f7e7c06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e82d70700 1 --2- 172.21.15.79:0/2073522477 >> v2:172.21.15.156:3300/0 conn(0x7f7e7c15c1f0 0x7f7e7c06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e82d70700 1 -- 172.21.15.79:0/2073522477 >> v1:172.21.15.79:6789/0 conn(0x7f7e7c071fe0 legacy=0x7f7e7c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e82d70700 1 -- 172.21.15.79:0/2073522477 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c071be0 con 0x7f7e7c069d50 2023-06-08T03:51:25.008 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8256f700 1 --2- 172.21.15.79:0/2073522477 >> v2:172.21.15.156:3300/0 conn(0x7f7e7c15c1f0 0x7f7e7c06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e82d70700 1 --2- 172.21.15.79:0/2073522477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c06a1d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c070d60 tx=0x7f7e7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64b5dfec2e570e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8156d700 1 -- 172.21.15.79:0/2073522477 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400e070 con 0x7f7e7c069d50 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8156d700 1 -- 172.21.15.79:0/2073522477 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7400bb20 con 0x7f7e7c069d50 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e8156d700 1 -- 172.21.15.79:0/2073522477 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74012430 con 0x7f7e7c069d50 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e847d3700 1 -- 172.21.15.79:0/2073522477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 msgr2=0x7f7e7c06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.002+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2073522477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c06a1d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c070d60 tx=0x7f7e7400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:25.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.003+0000 7f7e847d3700 1 -- 172.21.15.79:0/2073522477 shutdown_connections 2023-06-08T03:51:25.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.003+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2073522477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c06a1d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.003+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2073522477 >> v2:172.21.15.156:3300/0 conn(0x7f7e7c15c1f0 0x7f7e7c06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.003+0000 7f7e847d3700 1 -- 172.21.15.79:0/2073522477 >> 172.21.15.79:0/2073522477 conn(0x7f7e7c0b1d30 msgr2=0x7f7e7c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:25.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.003+0000 7f7e847d3700 1 -- 172.21.15.79:0/2073522477 shutdown_connections 2023-06-08T03:51:25.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.003+0000 7f7e847d3700 1 -- 172.21.15.79:0/2073522477 wait complete. 2023-06-08T03:51:25.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.004+0000 7f7e847d3700 1 Processor -- start 2023-06-08T03:51:25.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.004+0000 7f7e847d3700 1 -- start start 2023-06-08T03:51:25.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c071fe0 0x7f7e7c0c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e7c15c1f0 0x7f7e7c0c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c41d0 con 0x7f7e7c069d50 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c67c0 con 0x7f7e7c071fe0 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0c6aa0 con 0x7f7e7c15c1f0 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c0c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36632/0 (socket says 172.21.15.79:36632) 2023-06-08T03:51:25.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 -- 172.21.15.79:0/2100558012 learned_addr learned my addr 172.21.15.79:0/2100558012 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e81d6e700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c071fe0 0x7f7e7c0c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e82d70700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e7c15c1f0 0x7f7e7c0c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 -- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e7c15c1f0 msgr2=0x7f7e7c0c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e7c15c1f0 0x7f7e7c0c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 -- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c071fe0 msgr2=0x7f7e7c0c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c071fe0 0x7f7e7c0c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.014 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 -- 172.21.15.79:0/2100558012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e74009a70 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e8256f700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c0c5bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00e810 tx=0x7f7e6c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00c020 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c004250 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c0c6d80 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0036a0 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.005+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c0c6130 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.006+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c073e90 con 0x7f7e7c069d50 2023-06-08T03:51:25.015 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.007+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e6c003800 con 0x7f7e7c069d50 2023-06-08T03:51:25.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.007+0000 7f7e72ffd700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7e6006acb0 0x7f7e6006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.007+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7e6c089180 con 0x7f7e7c069d50 2023-06-08T03:51:25.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.007+0000 7f7e81d6e700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7e6006acb0 0x7f7e6006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.008+0000 7f7e81d6e700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7e6006acb0 0x7f7e6006d160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c1e7f10 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.016 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.011+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e6c053fd0 con 0x7f7e7c069d50 2023-06-08T03:51:25.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.196+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e7c06ed70 con 0x7f7e7c069d50 2023-06-08T03:51:25.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.197+0000 7f7e72ffd700 1 -- 172.21.15.79:0/2100558012 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7e6c0575f0 con 0x7f7e7c069d50 2023-06-08T03:51:25.201 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:25.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7e6006acb0 msgr2=0x7f7e6006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7e6006acb0 0x7f7e6006d160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c1e7f10 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2023-06-08T03:51:25.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 msgr2=0x7f7e7c0c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c0c5bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00e810 tx=0x7f7e6c00ebd0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 shutdown_connections 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7e6006acb0 0x7f7e6006d160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e7c15c1f0 0x7f7e7c0c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e7c069d50 0x7f7e7c0c5bf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 --2- 172.21.15.79:0/2100558012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e7c071fe0 0x7f7e7c0c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 >> 172.21.15.79:0/2100558012 conn(0x7f7e7c0b1d30 msgr2=0x7f7e7c0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:25.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 shutdown_connections 2023-06-08T03:51:25.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.199+0000 7f7e847d3700 1 -- 172.21.15.79:0/2100558012 wait complete. 2023-06-08T03:51:25.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:25.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.332+0000 7fefa7d8a700 1 Processor -- start 2023-06-08T03:51:25.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.333+0000 7fefa7d8a700 1 -- start start 2023-06-08T03:51:25.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.333+0000 7fefa7d8a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fefa006d8a0 0x7fefa0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.333+0000 7fefa7d8a700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa00cd140 con 0x7fefa0069d50 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.333+0000 7fefa7d8a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa0071110 con 0x7fefa015c1f0 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.333+0000 7fefa7d8a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa0071be0 con 0x7fefa006d8a0 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa6327700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fefa0069d50 0x7fefa00cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60120/0 (socket says 172.21.15.79:60120) 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa6327700 1 -- 172.21.15.79:0/1626523116 learned_addr learned my addr 172.21.15.79:0/1626523116 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa5325700 1 --2- 172.21.15.79:0/1626523116 >> v2:172.21.15.156:3300/0 conn(0x7fefa006d8a0 0x7fefa0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154325606 0 0) 0x7fefa00cd140 con 0x7fefa0069d50 2023-06-08T03:51:25.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef8c0035b0 con 0x7fefa0069d50 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2770913847 0 0) 0x7fef8c0035b0 con 0x7fefa0069d50 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 >> v2:172.21.15.156:3300/0 conn(0x7fefa006d8a0 msgr2=0x7fefa0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 --2- 172.21.15.79:0/1626523116 >> v2:172.21.15.156:3300/0 conn(0x7fefa006d8a0 0x7fefa0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 >> v1:172.21.15.156:6789/0 conn(0x7fefa015c1f0 legacy=0x7fefa00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa00cd390 con 0x7fefa0069d50 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fef980019f0 con 0x7fefa0069d50 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 >> v1:172.21.15.79:6789/0 conn(0x7fefa0069d50 legacy=0x7fefa00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fef8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef8c004850 0x7fef8c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef8c005320 0x7fef8c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0071be0 con 0x7fef8c003a00 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c00c250 con 0x7fef8c004850 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c00c410 con 0x7fef8c005320 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa5b26700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fef8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa5325700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef8c004850 0x7fef8c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.334+0000 7fefa6327700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef8c005320 0x7fef8c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5b26700 1 -- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef8c005320 msgr2=0x7fef8c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5b26700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef8c005320 0x7fef8c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5b26700 1 -- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef8c004850 msgr2=0x7fef8c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5b26700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef8c004850 0x7fef8c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5b26700 1 -- 172.21.15.79:0/1626523116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0035b0 con 0x7fef8c003a00 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa6327700 1 -- 172.21.15.79:0/1626523116 reap_dead start 2023-06-08T03:51:25.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5325700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef8c004850 0x7fef8c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa5b26700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fef8c003e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fef900017c0 tx=0x7fef8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eae1cfe5731156 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900052d0 con 0x7fef8c003a00 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90005430 con 0x7fef8c003a00 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa4b24700 1 -- 172.21.15.79:0/1626523116 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900038c0 con 0x7fef8c003a00 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 -- 172.21.15.79:0/1626523116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 msgr2=0x7fef8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fef8c003e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fef900017c0 tx=0x7fef8c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:25.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 -- 172.21.15.79:0/1626523116 shutdown_connections 2023-06-08T03:51:25.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/1626523116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fef8c003e10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 -- 172.21.15.79:0/1626523116 >> 172.21.15.79:0/1626523116 conn(0x7fefa00b1d30 msgr2=0x7fefa006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:25.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 -- 172.21.15.79:0/1626523116 shutdown_connections 2023-06-08T03:51:25.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.335+0000 7fefa7d8a700 1 -- 172.21.15.79:0/1626523116 wait complete. 2023-06-08T03:51:25.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 Processor -- start 2023-06-08T03:51:25.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 -- start start 2023-06-08T03:51:25.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fefa01559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fefa0158860 0x7fefa0158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefa01566e0 0x7fefa0156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa01e9400 con 0x7fef8c003a00 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0141e70 con 0x7fefa01566e0 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa7d8a700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0157090 con 0x7fefa0158860 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa5b26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fefa01559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa5b26700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fefa01559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36652/0 (socket says 172.21.15.79:36652) 2023-06-08T03:51:25.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.337+0000 7fefa5b26700 1 -- 172.21.15.79:0/3460967886 learned_addr learned my addr 172.21.15.79:0/3460967886 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5325700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fefa0158860 0x7fefa0158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa6327700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefa01566e0 0x7fefa0156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5b26700 1 -- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fefa0158860 msgr2=0x7fefa0158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5b26700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fefa0158860 0x7fefa0158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5b26700 1 -- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefa01566e0 msgr2=0x7fefa0156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5b26700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefa01566e0 0x7fefa0156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5b26700 1 -- 172.21.15.79:0/3460967886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0141b20 con 0x7fef8c003a00 2023-06-08T03:51:25.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5325700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fefa0158860 0x7fefa0158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa6327700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefa01566e0 0x7fefa0156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa5b26700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fefa01559e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fef90001db0 tx=0x7fef90001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90001290 con 0x7fef8c003a00 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90007040 con 0x7fef8c003a00 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa0159300 con 0x7fef8c003a00 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900073d0 con 0x7fef8c003a00 2023-06-08T03:51:25.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.338+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa01598c0 con 0x7fef8c003a00 2023-06-08T03:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.339+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa0159a50 con 0x7fef8c003a00 2023-06-08T03:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.339+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef90007530 con 0x7fef8c003a00 2023-06-08T03:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.340+0000 7fef967fc700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fef8406ace0 0x7fef8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.340+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fef9008a0e0 con 0x7fef8c003a00 2023-06-08T03:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.340+0000 7fefa5325700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fef8406ace0 0x7fef8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.340+0000 7fefa5325700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fef8406ace0 0x7fef8406d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefa00c65f0 tx=0x7fef9c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.344+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef90054f30 con 0x7fef8c003a00 2023-06-08T03:51:25.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.528+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fefa0073e90 con 0x7fef8c003a00 2023-06-08T03:51:25.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.529+0000 7fef967fc700 1 -- 172.21.15.79:0/3460967886 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fef90058550 con 0x7fef8c003a00 2023-06-08T03:51:25.533 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fef8406ace0 msgr2=0x7fef8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fef8406ace0 0x7fef8406d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefa00c65f0 tx=0x7fef9c008040 comp rx=0 tx=0).stop 2023-06-08T03:51:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 msgr2=0x7fefa01559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fefa01559e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fef90001db0 tx=0x7fef90001120 comp rx=0 tx=0).stop 2023-06-08T03:51:25.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 shutdown_connections 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fefa01566e0 0x7fefa0156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef8c003a00 0x7fefa01559e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fef8406ace0 0x7fef8406d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 --2- 172.21.15.79:0/3460967886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fefa0158860 0x7fefa0158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 >> 172.21.15.79:0/3460967886 conn(0x7fefa00b1d30 msgr2=0x7fefa00cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 shutdown_connections 2023-06-08T03:51:25.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.531+0000 7fefa7d8a700 1 -- 172.21.15.79:0/3460967886 wait complete. 2023-06-08T03:51:25.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.664+0000 7f16c8b9d700 1 Processor -- start 2023-06-08T03:51:25.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.664+0000 7f16c8b9d700 1 -- start start 2023-06-08T03:51:25.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.665+0000 7f16c8b9d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16c415e3e0 0x7f16c40cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.665+0000 7f16c8b9d700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c406db90 con 0x7f16c406c050 2023-06-08T03:51:25.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.665+0000 7f16c8b9d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40ca400 con 0x7f16c40715a0 2023-06-08T03:51:25.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.665+0000 7f16c8b9d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c415a290 con 0x7f16c415e3e0 2023-06-08T03:51:25.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.665+0000 7f16c1d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f16c406c050 0x7f16c40ca890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60138/0 (socket says 172.21.15.79:60138) 2023-06-08T03:51:25.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.665+0000 7f16c1d9b700 1 -- 172.21.15.79:0/608635005 learned_addr learned my addr 172.21.15.79:0/608635005 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c2d9d700 1 --2- 172.21.15.79:0/608635005 >> v2:172.21.15.156:3300/0 conn(0x7f16c415e3e0 0x7f16c40cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c159a700 1 -- 172.21.15.79:0/608635005 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290201219 0 0) 0x7f16c406db90 con 0x7f16c406c050 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c159a700 1 -- 172.21.15.79:0/608635005 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16ac0035b0 con 0x7f16c406c050 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c159a700 1 -- 172.21.15.79:0/608635005 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339862009 0 0) 0x7f16ac0035b0 con 0x7f16c406c050 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c2d9d700 1 -- 172.21.15.79:0/608635005 >> v1:172.21.15.156:6789/0 conn(0x7f16c40715a0 legacy=0x7f16c4071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c2d9d700 1 -- 172.21.15.79:0/608635005 >> v1:172.21.15.79:6789/0 conn(0x7f16c406c050 legacy=0x7f16c40ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c2d9d700 1 -- 172.21.15.79:0/608635005 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40ab0d0 con 0x7f16c415e3e0 2023-06-08T03:51:25.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c2d9d700 1 --2- 172.21.15.79:0/608635005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40cafa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009d90 tx=0x7f16bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59dfaf31f1509d3d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c159a700 1 -- 172.21.15.79:0/608635005 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00e070 con 0x7f16c415e3e0 2023-06-08T03:51:25.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c159a700 1 -- 172.21.15.79:0/608635005 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc003a20 con 0x7f16c415e3e0 2023-06-08T03:51:25.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.666+0000 7f16c159a700 1 -- 172.21.15.79:0/608635005 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4001670 con 0x7f16c415e3e0 2023-06-08T03:51:25.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 -- 172.21.15.79:0/608635005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 msgr2=0x7f16c40cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/608635005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40cafa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009d90 tx=0x7f16bc00b330 comp rx=0 tx=0).stop 2023-06-08T03:51:25.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 -- 172.21.15.79:0/608635005 shutdown_connections 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/608635005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40cafa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 -- 172.21.15.79:0/608635005 >> 172.21.15.79:0/608635005 conn(0x7f16c40ce4e0 msgr2=0x7f16c40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 -- 172.21.15.79:0/608635005 shutdown_connections 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.667+0000 7f16c8b9d700 1 -- 172.21.15.79:0/608635005 wait complete. 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.668+0000 7f16c8b9d700 1 Processor -- start 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.668+0000 7f16c8b9d700 1 -- start start 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c8b9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 0x7f16c40ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c8b9d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16c40715a0 0x7f16c40b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c8b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c8b9d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40c70b0 con 0x7f16c406c050 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c8b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40c55b0 con 0x7f16c415e3e0 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c8b9d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4096f60 con 0x7f16c40715a0 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 0x7f16c40ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 0x7f16c40ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36666/0 (socket says 172.21.15.79:36666) 2023-06-08T03:51:25.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 -- 172.21.15.79:0/3212952167 learned_addr learned my addr 172.21.15.79:0/3212952167 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c1d9b700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16c40715a0 0x7f16c40b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c2d9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 -- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16c40715a0 msgr2=0x7f16c40b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16c40715a0 0x7f16c40b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 -- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 msgr2=0x7f16c40b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.669+0000 7f16c259c700 1 -- 172.21.15.79:0/3212952167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc009a70 con 0x7f16c406c050 2023-06-08T03:51:25.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.670+0000 7f16c259c700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 0x7f16c40ccb60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c6e0 tx=0x7f16b800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.670+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80052d0 con 0x7f16c406c050 2023-06-08T03:51:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.670+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8005430 con 0x7f16c406c050 2023-06-08T03:51:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.670+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c40c5800 con 0x7f16c406c050 2023-06-08T03:51:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.670+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80038c0 con 0x7f16c406c050 2023-06-08T03:51:25.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.670+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c4096bf0 con 0x7f16c406c050 2023-06-08T03:51:25.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.671+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c4073e90 con 0x7f16c406c050 2023-06-08T03:51:25.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.671+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16b8003a20 con 0x7f16c406c050 2023-06-08T03:51:25.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.671+0000 7f16b2ffd700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f16a406ace0 0x7f16a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:25.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.672+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f16b80889f0 con 0x7f16c406c050 2023-06-08T03:51:25.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.672+0000 7f16c1d9b700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f16a406ace0 0x7f16a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:25.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.672+0000 7f16c1d9b700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f16a406ace0 0x7f16a406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16c40c5070 tx=0x7f16b4001da0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:25.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.676+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16b8053840 con 0x7f16c406c050 2023-06-08T03:51:25.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.879+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f16c41e67d0 con 0x7f16c406c050 2023-06-08T03:51:25.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.879+0000 7f16b2ffd700 1 -- 172.21.15.79:0/3212952167 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f16b8056e60 con 0x7f16c406c050 2023-06-08T03:51:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f16a406ace0 msgr2=0x7f16a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f16a406ace0 0x7f16a406d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16c40c5070 tx=0x7f16b4001da0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 msgr2=0x7f16c40ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 0x7f16c40ccb60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c6e0 tx=0x7f16b800c9f0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 shutdown_connections 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16c415e3e0 0x7f16c40b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c406c050 0x7f16c40ccb60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f16a406ace0 0x7f16a406d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 --2- 172.21.15.79:0/3212952167 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16c40715a0 0x7f16c40b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 >> 172.21.15.79:0/3212952167 conn(0x7f16c40ce4e0 msgr2=0x7f16c40ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 shutdown_connections 2023-06-08T03:51:25.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:25.882+0000 7f16c8b9d700 1 -- 172.21.15.79:0/3212952167 wait complete. 2023-06-08T03:51:25.888 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:25.898 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:51:25.898 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:25.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:26.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.071+0000 7fbdb89ab700 1 Processor -- start 2023-06-08T03:51:26.076 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.071+0000 7fbdb89ab700 1 -- start start 2023-06-08T03:51:26.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.072+0000 7fbdb89ab700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbdb40c99c0 0x7fbdb415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.072+0000 7fbdb89ab700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb40ceac0 con 0x7fbdb40ccd40 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.072+0000 7fbdb89ab700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb40ce0c0 con 0x7fbdb415b080 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.072+0000 7fbdb89ab700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb40cd4b0 con 0x7fbdb40c99c0 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb2d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbdb40ccd40 0x7fbdb406a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60154/0 (socket says 172.21.15.79:60154) 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb2d9d700 1 -- 172.21.15.79:0/4126649843 learned_addr learned my addr 172.21.15.79:0/4126649843 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb1d9b700 1 --2- 172.21.15.79:0/4126649843 >> v2:172.21.15.156:3300/0 conn(0x7fbdb40c99c0 0x7fbdb415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059250392 0 0) 0x7fbdac001490 con 0x7fbdb40ccd40 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd9c0035b0 con 0x7fbdb40ccd40 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3533680242 0 0) 0x7fbd9c0035b0 con 0x7fbdb40ccd40 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 >> v2:172.21.15.156:3300/0 conn(0x7fbdb40c99c0 msgr2=0x7fbdb415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 --2- 172.21.15.79:0/4126649843 >> v2:172.21.15.156:3300/0 conn(0x7fbdb40c99c0 0x7fbdb415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 >> v1:172.21.15.156:6789/0 conn(0x7fbdb415b080 legacy=0x7fbdb40c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb40cd790 con 0x7fbdb40ccd40 2023-06-08T03:51:26.080 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fbdac001b80 con 0x7fbdb40ccd40 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 >> v1:172.21.15.79:6789/0 conn(0x7fbdb40ccd40 legacy=0x7fbdb406a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd9c003a70 0x7fbd9c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004850 0x7fbd9c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c005200 0x7fbd9c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40cd4b0 con 0x7fbd9c003a70 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c00c0e0 con 0x7fbd9c004850 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c00c2a0 con 0x7fbd9c005200 2023-06-08T03:51:26.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb259c700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd9c003a70 0x7fbd9c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb2d9d700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c005200 0x7fbd9c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.073+0000 7fbdb1d9b700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004850 0x7fbd9c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb259c700 1 -- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c005200 msgr2=0x7fbd9c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb259c700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c005200 0x7fbd9c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb259c700 1 -- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004850 msgr2=0x7fbd9c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb259c700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd9c004850 0x7fbd9c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb259c700 1 -- 172.21.15.79:0/4126649843 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0035b0 con 0x7fbd9c003a70 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb2d9d700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c005200 0x7fbd9c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb2d9d700 1 -- 172.21.15.79:0/4126649843 reap_dead start 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb259c700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd9c003a70 0x7fbd9c003e80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbda40017c0 tx=0x7fbd9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1285e682dfa9ff85 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda40052d0 con 0x7fbd9c003a70 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda4005430 con 0x7fbd9c003a70 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb159a700 1 -- 172.21.15.79:0/4126649843 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda40038c0 con 0x7fbd9c003a70 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb89ab700 1 -- 172.21.15.79:0/4126649843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd9c003a70 msgr2=0x7fbd9c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd9c003a70 0x7fbd9c003e80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbda40017c0 tx=0x7fbd9c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:26.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.074+0000 7fbdb89ab700 1 -- 172.21.15.79:0/4126649843 shutdown_connections 2023-06-08T03:51:26.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.075+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/4126649843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd9c003a70 0x7fbd9c003e80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.075+0000 7fbdb89ab700 1 -- 172.21.15.79:0/4126649843 >> 172.21.15.79:0/4126649843 conn(0x7fbdb40cf8f0 msgr2=0x7fbdb40cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:26.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.075+0000 7fbdb89ab700 1 -- 172.21.15.79:0/4126649843 shutdown_connections 2023-06-08T03:51:26.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.075+0000 7fbdb89ab700 1 -- 172.21.15.79:0/4126649843 wait complete. 2023-06-08T03:51:26.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.076+0000 7fbdb89ab700 1 Processor -- start 2023-06-08T03:51:26.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.076+0000 7fbdb89ab700 1 -- start start 2023-06-08T03:51:26.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb89ab700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c003a70 0x7fbdb40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb89ab700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb414b370 0x7fbdb4151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb89ab700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 0x7fbdb40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb89ab700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40b4120 con 0x7fbdb414ae20 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb89ab700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb41503f0 con 0x7fbdb414b370 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb89ab700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4150690 con 0x7fbd9c003a70 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 0x7fbdb40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 0x7fbdb40c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36690/0 (socket says 172.21.15.79:36690) 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 -- 172.21.15.79:0/2397478937 learned_addr learned my addr 172.21.15.79:0/2397478937 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:26.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb1d9b700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb414b370 0x7fbdb4151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb259c700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c003a70 0x7fbdb40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 -- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c003a70 msgr2=0x7fbdb40b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c003a70 0x7fbdb40b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 -- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb414b370 msgr2=0x7fbdb4151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb414b370 0x7fbdb4151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 -- 172.21.15.79:0/2397478937 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4150970 con 0x7fbdb414ae20 2023-06-08T03:51:26.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbdb2d9d700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 0x7fbdb40c7b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbdac001330 tx=0x7fbdac00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00ec70 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.077+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac00edd0 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.078+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac013660 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.078+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb4150c50 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.078+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb41516b0 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.078+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb4073e90 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.079+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdac0137c0 con 0x7fbdb414ae20 2023-06-08T03:51:26.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.079+0000 7fbda2ffd700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd9406ace0 0x7fbd9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.079+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbdac089a90 con 0x7fbdb414ae20 2023-06-08T03:51:26.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.080+0000 7fbdb259c700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd9406ace0 0x7fbd9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.080+0000 7fbdb259c700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd9406ace0 0x7fbd9406d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbda4015040 tx=0x7fbda4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.084+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbdac054850 con 0x7fbdb414ae20 2023-06-08T03:51:26.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.272+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdb40ceac0 con 0x7fbdb414ae20 2023-06-08T03:51:26.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.273+0000 7fbda2ffd700 1 -- 172.21.15.79:0/2397478937 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7fbdb40ceac0 con 0x7fbdb414ae20 2023-06-08T03:51:26.278 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:26.278 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:26.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd9406ace0 msgr2=0x7fbd9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd9406ace0 0x7fbd9406d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbda4015040 tx=0x7fbda4009030 comp rx=0 tx=0).stop 2023-06-08T03:51:26.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 msgr2=0x7fbdb40c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 0x7fbdb40c7b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbdac001330 tx=0x7fbdac00e440 comp rx=0 tx=0).stop 2023-06-08T03:51:26.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 shutdown_connections 2023-06-08T03:51:26.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdb414ae20 0x7fbdb40c7b90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd9406ace0 0x7fbd9406d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd9c003a70 0x7fbdb40b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 --2- 172.21.15.79:0/2397478937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbdb414b370 0x7fbdb4151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 >> 172.21.15.79:0/2397478937 conn(0x7fbdb40cf8f0 msgr2=0x7fbdb4069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:26.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 shutdown_connections 2023-06-08T03:51:26.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.275+0000 7fbdb89ab700 1 -- 172.21.15.79:0/2397478937 wait complete. 2023-06-08T03:51:26.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:26.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:26.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.411+0000 7f289dfa0700 1 Processor -- start 2023-06-08T03:51:26.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.411+0000 7f289dfa0700 1 -- start start 2023-06-08T03:51:26.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f289dfa0700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f289806cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f289dfa0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f289806d480 0x7f28980c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f289dfa0700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28980c9070 0x7f289815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f289dfa0700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28980c94f0 con 0x7f28980c9070 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f289dfa0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28980ce330 con 0x7f289806d480 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f289dfa0700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28980abaa0 con 0x7f28980cdf60 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28980c9070 0x7f289815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28980c9070 0x7f289815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36698/0 (socket says 172.21.15.79:36698) 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 -- 172.21.15.79:0/3463760808 learned_addr learned my addr 172.21.15.79:0/3463760808 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f28977fe700 1 --2- 172.21.15.79:0/3463760808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f289806cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2896ffd700 1 --2- 172.21.15.79:0/3463760808 >> v2:172.21.15.156:3300/0 conn(0x7f289806d480 0x7f28980c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 -- 172.21.15.79:0/3463760808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 msgr2=0x7f289806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 --2- 172.21.15.79:0/3463760808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f289806cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 -- 172.21.15.79:0/3463760808 >> v2:172.21.15.156:3300/0 conn(0x7f289806d480 msgr2=0x7f28980c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 --2- 172.21.15.79:0/3463760808 >> v2:172.21.15.156:3300/0 conn(0x7f289806d480 0x7f28980c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 -- 172.21.15.79:0/3463760808 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289806d950 con 0x7f28980c9070 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.412+0000 7f2897fff700 1 --2- 172.21.15.79:0/3463760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28980c9070 0x7f289815c1a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f289806f930 tx=0x7f289000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8eb9b59e4da7dc server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f28967fc700 1 -- 172.21.15.79:0/3463760808 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000e070 con 0x7f28980c9070 2023-06-08T03:51:26.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f28967fc700 1 -- 172.21.15.79:0/3463760808 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289000b9c0 con 0x7f28980c9070 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f28967fc700 1 -- 172.21.15.79:0/3463760808 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289000bd70 con 0x7f28980c9070 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 -- 172.21.15.79:0/3463760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28980c9070 msgr2=0x7f289815c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 --2- 172.21.15.79:0/3463760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28980c9070 0x7f289815c1a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f289806f930 tx=0x7f289000b330 comp rx=0 tx=0).stop 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 -- 172.21.15.79:0/3463760808 shutdown_connections 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 --2- 172.21.15.79:0/3463760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28980c9070 0x7f289815c1a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 --2- 172.21.15.79:0/3463760808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f289806cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 --2- 172.21.15.79:0/3463760808 >> v2:172.21.15.156:3300/0 conn(0x7f289806d480 0x7f28980c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 -- 172.21.15.79:0/3463760808 >> 172.21.15.79:0/3463760808 conn(0x7f28980ce4e0 msgr2=0x7f28980ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:26.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 -- 172.21.15.79:0/3463760808 shutdown_connections 2023-06-08T03:51:26.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.413+0000 7f289dfa0700 1 -- 172.21.15.79:0/3463760808 wait complete. 2023-06-08T03:51:26.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.414+0000 7f289dfa0700 1 Processor -- start 2023-06-08T03:51:26.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 -- start start 2023-06-08T03:51:26.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 0x7f289809bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28980c9070 0x7f28980c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f28980982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2898149880 con 0x7f289806d480 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28980c4500 con 0x7f28980c9070 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f289dfa0700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2898148f20 con 0x7f28980cdf60 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f28977fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 0x7f289809bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f28977fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 0x7f289809bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36702/0 (socket says 172.21.15.79:36702) 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.415+0000 7f28977fe700 1 -- 172.21.15.79:0/2198400664 learned_addr learned my addr 172.21.15.79:0/2198400664 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f2897fff700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f28980982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f2896ffd700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28980c9070 0x7f28980c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f28977fe700 1 -- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 msgr2=0x7f28980982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f28977fe700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f28980982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f28977fe700 1 -- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28980c9070 msgr2=0x7f28980c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f28977fe700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28980c9070 0x7f28980c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f28977fe700 1 -- 172.21.15.79:0/2198400664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890009b30 con 0x7f289806d480 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f28977fe700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 0x7f289809bcf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f288c00d800 tx=0x7f288c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c0125f0 con 0x7f289806d480 2023-06-08T03:51:26.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c012750 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c012b00 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289809c230 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.416+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28981494a0 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.417+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2898073e90 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.417+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f288c012c60 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.418+0000 7f287ffff700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f288406ace0 0x7f288406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.418+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f288c089af0 con 0x7f289806d480 2023-06-08T03:51:26.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.418+0000 7f2896ffd700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f288406ace0 0x7f288406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.418+0000 7f2896ffd700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f288406ace0 0x7f288406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28980c89d0 tx=0x7f2888008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.422+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f288c0548b0 con 0x7f289806d480 2023-06-08T03:51:26.609 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.605+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f289806d260 con 0x7f289806d480 2023-06-08T03:51:26.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.606+0000 7f287ffff700 1 -- 172.21.15.79:0/2198400664 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f288c057ed0 con 0x7f289806d480 2023-06-08T03:51:26.610 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:26.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f288406ace0 msgr2=0x7f288406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f288406ace0 0x7f288406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f28980c89d0 tx=0x7f2888008040 comp rx=0 tx=0).stop 2023-06-08T03:51:26.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 msgr2=0x7f289809bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 0x7f289809bcf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f288c00d800 tx=0x7f288c00dbc0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 shutdown_connections 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f288406ace0 0x7f288406d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28980cdf60 0x7f28980982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f289806d480 0x7f289809bcf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 --2- 172.21.15.79:0/2198400664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28980c9070 0x7f28980c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 >> 172.21.15.79:0/2198400664 conn(0x7f28980ce4e0 msgr2=0x7f289815b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 shutdown_connections 2023-06-08T03:51:26.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.608+0000 7f289dfa0700 1 -- 172.21.15.79:0/2198400664 wait complete. 2023-06-08T03:51:26.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:26.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.742+0000 7fab444fc700 1 Processor -- start 2023-06-08T03:51:26.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.742+0000 7fab444fc700 1 -- start start 2023-06-08T03:51:26.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab444fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab3c15c1f0 0x7fab3c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab444fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fab3c069d50 0x7fab3c06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab444fc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3c0cd140 con 0x7fab3c071fe0 2023-06-08T03:51:26.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab444fc700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c06a710 con 0x7fab3c15c1f0 2023-06-08T03:51:26.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab444fc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c071110 con 0x7fab3c069d50 2023-06-08T03:51:26.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab3c15c1f0 0x7fab3c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab3c15c1f0 0x7fab3c06d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36708/0 (socket says 172.21.15.79:36708) 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 -- 172.21.15.79:0/2945876564 learned_addr learned my addr 172.21.15.79:0/2945876564 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42a99700 1 --2- 172.21.15.79:0/2945876564 >> v2:172.21.15.156:3300/0 conn(0x7fab3c069d50 0x7fab3c06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 -- 172.21.15.79:0/2945876564 >> v2:172.21.15.156:3300/0 conn(0x7fab3c069d50 msgr2=0x7fab3c06a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 --2- 172.21.15.79:0/2945876564 >> v2:172.21.15.156:3300/0 conn(0x7fab3c069d50 0x7fab3c06a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 -- 172.21.15.79:0/2945876564 >> v1:172.21.15.156:6789/0 conn(0x7fab3c071fe0 legacy=0x7fab3c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42298700 1 -- 172.21.15.79:0/2945876564 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c071be0 con 0x7fab3c15c1f0 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.743+0000 7fab42a99700 1 --2- 172.21.15.79:0/2945876564 >> v2:172.21.15.156:3300/0 conn(0x7fab3c069d50 0x7fab3c06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:26.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab42298700 1 --2- 172.21.15.79:0/2945876564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c06d8a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fab3c070e40 tx=0x7fab2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c390bb4c97b1a2b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab41296700 1 -- 172.21.15.79:0/2945876564 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c004230 con 0x7fab3c15c1f0 2023-06-08T03:51:26.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab41296700 1 -- 172.21.15.79:0/2945876564 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c004390 con 0x7fab3c15c1f0 2023-06-08T03:51:26.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab41296700 1 -- 172.21.15.79:0/2945876564 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0036a0 con 0x7fab3c15c1f0 2023-06-08T03:51:26.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 -- 172.21.15.79:0/2945876564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 msgr2=0x7fab3c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 --2- 172.21.15.79:0/2945876564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c06d8a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fab3c070e40 tx=0x7fab2c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 -- 172.21.15.79:0/2945876564 shutdown_connections 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 --2- 172.21.15.79:0/2945876564 >> v2:172.21.15.156:3300/0 conn(0x7fab3c069d50 0x7fab3c06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 --2- 172.21.15.79:0/2945876564 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c06d8a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 -- 172.21.15.79:0/2945876564 >> 172.21.15.79:0/2945876564 conn(0x7fab3c0b1d30 msgr2=0x7fab3c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 -- 172.21.15.79:0/2945876564 shutdown_connections 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.744+0000 7fab444fc700 1 -- 172.21.15.79:0/2945876564 wait complete. 2023-06-08T03:51:26.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 Processor -- start 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 -- start start 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c069d50 0x7fab3c1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c071fe0 0x7fab3c0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0c80a0 con 0x7fab3c15c1f0 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0c8380 con 0x7fab3c069d50 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab444fc700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0c8660 con 0x7fab3c071fe0 2023-06-08T03:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab42a99700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab42a99700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36718/0 (socket says 172.21.15.79:36718) 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.746+0000 7fab42a99700 1 -- 172.21.15.79:0/3815663500 learned_addr learned my addr 172.21.15.79:0/3815663500 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42298700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c069d50 0x7fab3c1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab41a97700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c071fe0 0x7fab3c0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42a99700 1 -- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c071fe0 msgr2=0x7fab3c0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42a99700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c071fe0 0x7fab3c0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42a99700 1 -- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c069d50 msgr2=0x7fab3c1f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42a99700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c069d50 0x7fab3c1f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42a99700 1 -- 172.21.15.79:0/3815663500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009550 con 0x7fab3c15c1f0 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab42a99700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c15c5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fab3400cd50 tx=0x7fab3400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400cea0 con 0x7fab3c15c1f0 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3400f480 con 0x7fab3c15c1f0 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c0c8940 con 0x7fab3c15c1f0 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400f830 con 0x7fab3c15c1f0 2023-06-08T03:51:26.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.747+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1f5140 con 0x7fab3c15c1f0 2023-06-08T03:51:26.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.748+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c0c8e60 con 0x7fab3c15c1f0 2023-06-08T03:51:26.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.749+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab3400b040 con 0x7fab3c15c1f0 2023-06-08T03:51:26.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.749+0000 7fab32ffd700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fab2006ace0 0x7fab2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:26.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.749+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fab34089d60 con 0x7fab3c15c1f0 2023-06-08T03:51:26.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.749+0000 7fab42298700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fab2006ace0 0x7fab2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:26.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.749+0000 7fab42298700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fab2006ace0 0x7fab2006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009130 tx=0x7fab2c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:26.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.753+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fab3c0c8e60 con 0x7fab3c15c1f0 2023-06-08T03:51:26.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.939+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab3c0c8e60 con 0x7fab3c15c1f0 2023-06-08T03:51:26.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.939+0000 7fab32ffd700 1 -- 172.21.15.79:0/3815663500 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab3c0c8e60 con 0x7fab3c15c1f0 2023-06-08T03:51:26.944 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:26.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fab2006ace0 msgr2=0x7fab2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fab2006ace0 0x7fab2006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009130 tx=0x7fab2c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:26.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 msgr2=0x7fab3c15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:26.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c15c5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fab3400cd50 tx=0x7fab3400a560 comp rx=0 tx=0).stop 2023-06-08T03:51:26.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 shutdown_connections 2023-06-08T03:51:26.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3c15c1f0 0x7fab3c15c5c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab3c069d50 0x7fab3c1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fab2006ace0 0x7fab2006d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 --2- 172.21.15.79:0/3815663500 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fab3c071fe0 0x7fab3c0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 >> 172.21.15.79:0/3815663500 conn(0x7fab3c0b1d30 msgr2=0x7fab3c0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 shutdown_connections 2023-06-08T03:51:26.947 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:26.942+0000 7fab444fc700 1 -- 172.21.15.79:0/3815663500 wait complete. 2023-06-08T03:51:26.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.075+0000 7f07f81b7700 1 Processor -- start 2023-06-08T03:51:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.075+0000 7f07f81b7700 1 -- start start 2023-06-08T03:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f81b7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07f015c1f0 0x7f07f006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f81b7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f0071fe0 0x7f07f00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f81b7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f07f00cf130 0x7f07f0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f81b7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f00cd140 con 0x7f07f015c1f0 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f81b7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f0071110 con 0x7f07f00cf130 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f81b7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0071be0 con 0x7f07f0071fe0 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07f015c1f0 0x7f07f006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07f015c1f0 0x7f07f006d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36722/0 (socket says 172.21.15.79:36722) 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 -- 172.21.15.79:0/3642263599 learned_addr learned my addr 172.21.15.79:0/3642263599 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5752700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f0071fe0 0x7f07f00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f6754700 1 --2- 172.21.15.79:0/3642263599 >> v2:172.21.15.156:3300/0 conn(0x7f07f00cf130 0x7f07f0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 -- 172.21.15.79:0/3642263599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f0071fe0 msgr2=0x7f07f00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f0071fe0 0x7f07f00cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 -- 172.21.15.79:0/3642263599 >> v2:172.21.15.156:3300/0 conn(0x7f07f00cf130 msgr2=0x7f07f0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 --2- 172.21.15.79:0/3642263599 >> v2:172.21.15.156:3300/0 conn(0x7f07f00cf130 0x7f07f0069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5f53700 1 -- 172.21.15.79:0/3642263599 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00cd390 con 0x7f07f015c1f0 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f6754700 1 --2- 172.21.15.79:0/3642263599 >> v2:172.21.15.156:3300/0 conn(0x7f07f00cf130 0x7f07f0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.076+0000 7f07f5752700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f0071fe0 0x7f07f00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f5f53700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f006d8a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f07f0070e40 tx=0x7f07e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da75696b3afb350 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f4f51700 1 -- 172.21.15.79:0/3642263599 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000b870 con 0x7f07f015c1f0 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f4f51700 1 -- 172.21.15.79:0/3642263599 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e000b9d0 con 0x7f07f015c1f0 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f4f51700 1 -- 172.21.15.79:0/3642263599 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000bd60 con 0x7f07f015c1f0 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 -- 172.21.15.79:0/3642263599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 msgr2=0x7f07f006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f006d8a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f07f0070e40 tx=0x7f07e0009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 -- 172.21.15.79:0/3642263599 shutdown_connections 2023-06-08T03:51:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 --2- 172.21.15.79:0/3642263599 >> v2:172.21.15.156:3300/0 conn(0x7f07f00cf130 0x7f07f0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f006d8a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 --2- 172.21.15.79:0/3642263599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f0071fe0 0x7f07f00cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 -- 172.21.15.79:0/3642263599 >> 172.21.15.79:0/3642263599 conn(0x7f07f00b1d30 msgr2=0x7f07f006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 -- 172.21.15.79:0/3642263599 shutdown_connections 2023-06-08T03:51:27.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.077+0000 7f07f81b7700 1 -- 172.21.15.79:0/3642263599 wait complete. 2023-06-08T03:51:27.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.078+0000 7f07f81b7700 1 Processor -- start 2023-06-08T03:51:27.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 -- start start 2023-06-08T03:51:27.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f0071fe0 0x7f07f0144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f00cf130 0x7f07f00c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f00c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01f1f20 con 0x7f07f015c1f0 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00b33c0 con 0x7f07f0071fe0 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f81b7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0144ea0 con 0x7f07f00cf130 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f6754700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f00c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f6754700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f00c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36732/0 (socket says 172.21.15.79:36732) 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.079+0000 7f07f6754700 1 -- 172.21.15.79:0/2093410736 learned_addr learned my addr 172.21.15.79:0/2093410736 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f5752700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f00cf130 0x7f07f00c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f5f53700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f0071fe0 0x7f07f0144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f6754700 1 -- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f00cf130 msgr2=0x7f07f00c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f6754700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f00cf130 0x7f07f00c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f6754700 1 -- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f0071fe0 msgr2=0x7f07f0144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f6754700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f0071fe0 0x7f07f0144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f6754700 1 -- 172.21.15.79:0/2093410736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e0009550 con 0x7f07f015c1f0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f5752700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f00cf130 0x7f07f00c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f6754700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f00c4d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f07e8007df0 tx=0x7f07e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e800ddd0 con 0x7f07f015c1f0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e800f040 con 0x7f07f015c1f0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8003c60 con 0x7f07f015c1f0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f00c5380 con 0x7f07f015c1f0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.080+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f00b2d70 con 0x7f07f015c1f0 2023-06-08T03:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.081+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f00b3570 con 0x7f07f015c1f0 2023-06-08T03:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.081+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07e8004030 con 0x7f07f015c1f0 2023-06-08T03:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.082+0000 7f07e67fc700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f07d406ace0 0x7f07d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.082+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f07e8089960 con 0x7f07f015c1f0 2023-06-08T03:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.082+0000 7f07f5f53700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f07d406ace0 0x7f07d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.082+0000 7f07f5f53700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f07d406ace0 0x7f07d406d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f07e0014040 tx=0x7f07e0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.086+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f07f00b3570 con 0x7f07f015c1f0 2023-06-08T03:51:27.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.273+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07f00b3530 con 0x7f07f015c1f0 2023-06-08T03:51:27.277 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.273+0000 7f07e67fc700 1 -- 172.21.15.79:0/2093410736 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07e8087e60 con 0x7f07f015c1f0 2023-06-08T03:51:27.278 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:27.279 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f07d406ace0 msgr2=0x7f07d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f07d406ace0 0x7f07d406d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f07e0014040 tx=0x7f07e0008030 comp rx=0 tx=0).stop 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 msgr2=0x7f07f00c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f00c4d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f07e8007df0 tx=0x7f07e800d3b0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 shutdown_connections 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f015c1f0 0x7f07f00c4d50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07f0071fe0 0x7f07f0144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.280 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f07d406ace0 0x7f07d406d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 --2- 172.21.15.79:0/2093410736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07f00cf130 0x7f07f00c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.275+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 >> 172.21.15.79:0/2093410736 conn(0x7f07f00b1d30 msgr2=0x7f07f015cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:27.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.276+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 shutdown_connections 2023-06-08T03:51:27.281 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.276+0000 7f07f81b7700 1 -- 172.21.15.79:0/2093410736 wait complete. 2023-06-08T03:51:27.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:27.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.409+0000 7f7dc77db700 1 Processor -- start 2023-06-08T03:51:27.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.410+0000 7f7dc77db700 1 -- start start 2023-06-08T03:51:27.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.410+0000 7f7dc77db700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc015c1f0 0x7f7dc006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.410+0000 7f7dc77db700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7dc0071fe0 0x7f7dc00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.410+0000 7f7dc77db700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc00cd140 con 0x7f7dc00cf130 2023-06-08T03:51:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.410+0000 7f7dc77db700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc0071110 con 0x7f7dc0071fe0 2023-06-08T03:51:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.410+0000 7f7dc77db700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0071be0 con 0x7f7dc015c1f0 2023-06-08T03:51:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc4d76700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7dc0071fe0 0x7f7dc00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc5d78700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7dc00cf130 0x7f7dc0069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60216/0 (socket says 172.21.15.79:60216) 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc5d78700 1 -- 172.21.15.79:0/3620442906 learned_addr learned my addr 172.21.15.79:0/3620442906 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc5577700 1 --2- 172.21.15.79:0/3620442906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc015c1f0 0x7f7dc006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7db7fff700 1 -- 172.21.15.79:0/3620442906 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140608881 0 0) 0x7f7dc00cd140 con 0x7f7dc00cf130 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7db7fff700 1 -- 172.21.15.79:0/3620442906 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dac0035b0 con 0x7f7dc00cf130 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc4d76700 1 -- 172.21.15.79:0/3620442906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc015c1f0 msgr2=0x7f7dc006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc4d76700 1 --2- 172.21.15.79:0/3620442906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc015c1f0 0x7f7dc006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc4d76700 1 -- 172.21.15.79:0/3620442906 >> v1:172.21.15.79:6789/0 conn(0x7f7dc00cf130 legacy=0x7f7dc0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc4d76700 1 -- 172.21.15.79:0/3620442906 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc00cd390 con 0x7f7dc0071fe0 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7dc4d76700 1 --2- 172.21.15.79:0/3620442906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc0071fe0 0x7f7dc00cebf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009840 tx=0x7f7dbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41eccfc5bcc769cf server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7db7fff700 1 -- 172.21.15.79:0/3620442906 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011830 con 0x7f7dc0071fe0 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7db7fff700 1 -- 172.21.15.79:0/3620442906 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc011990 con 0x7f7dc0071fe0 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.411+0000 7f7db7fff700 1 -- 172.21.15.79:0/3620442906 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011d20 con 0x7f7dc0071fe0 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 -- 172.21.15.79:0/3620442906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc0071fe0 msgr2=0x7f7dc00cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 --2- 172.21.15.79:0/3620442906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc0071fe0 0x7f7dc00cebf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009840 tx=0x7f7dbc009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:27.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 -- 172.21.15.79:0/3620442906 shutdown_connections 2023-06-08T03:51:27.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 --2- 172.21.15.79:0/3620442906 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc015c1f0 0x7f7dc006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 --2- 172.21.15.79:0/3620442906 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc0071fe0 0x7f7dc00cebf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 -- 172.21.15.79:0/3620442906 >> 172.21.15.79:0/3620442906 conn(0x7f7dc00b1d30 msgr2=0x7f7dc006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:27.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 -- 172.21.15.79:0/3620442906 shutdown_connections 2023-06-08T03:51:27.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.412+0000 7f7dc77db700 1 -- 172.21.15.79:0/3620442906 wait complete. 2023-06-08T03:51:27.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.413+0000 7f7dc77db700 1 Processor -- start 2023-06-08T03:51:27.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.413+0000 7f7dc77db700 1 -- start start 2023-06-08T03:51:27.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc77db700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc0071fe0 0x7f7dc0150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.421 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc77db700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc00cf130 0x7f7dc0151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc77db700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 0x7f7dc00c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc77db700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00b4660 con 0x7f7dc015c1f0 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc77db700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00c3dd0 con 0x7f7dc0071fe0 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc77db700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00c3af0 con 0x7f7dc00cf130 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 0x7f7dc00c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 0x7f7dc00c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36744/0 (socket says 172.21.15.79:36744) 2023-06-08T03:51:27.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 -- 172.21.15.79:0/2454085405 learned_addr learned my addr 172.21.15.79:0/2454085405 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5577700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc0071fe0 0x7f7dc0150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc4d76700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc00cf130 0x7f7dc0151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 -- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc00cf130 msgr2=0x7f7dc0151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc00cf130 0x7f7dc0151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 -- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc0071fe0 msgr2=0x7f7dc0150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc0071fe0 0x7f7dc0150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 -- 172.21.15.79:0/2454085405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0094a0 con 0x7f7dc015c1f0 2023-06-08T03:51:27.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5577700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc0071fe0 0x7f7dc0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7dc5d78700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 0x7f7dc00c5200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7db800caa0 tx=0x7f7db800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.414+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db800aa80 con 0x7f7dc015c1f0 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.415+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db800abe0 con 0x7f7dc015c1f0 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.415+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8012400 con 0x7f7dc015c1f0 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.415+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc00b3fa0 con 0x7f7dc015c1f0 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.415+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc01512f0 con 0x7f7dc015c1f0 2023-06-08T03:51:27.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.416+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc00b3e00 con 0x7f7dc015c1f0 2023-06-08T03:51:27.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.416+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7db801b3b0 con 0x7f7dc015c1f0 2023-06-08T03:51:27.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.416+0000 7f7db5ffb700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7da406ace0 0x7f7da406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.416+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7db8089460 con 0x7f7dc015c1f0 2023-06-08T03:51:27.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.417+0000 7f7dc5577700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7da406ace0 0x7f7da406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.417+0000 7f7dc5577700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7da406ace0 0x7f7da406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0151fc0 tx=0x7f7db0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.421+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7dc00b3e00 con 0x7f7dc015c1f0 2023-06-08T03:51:27.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.607+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7dc00b3e00 con 0x7f7dc015c1f0 2023-06-08T03:51:27.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.607+0000 7f7db5ffb700 1 -- 172.21.15.79:0/2454085405 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7dc00b3e00 con 0x7f7dc015c1f0 2023-06-08T03:51:27.611 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.609+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7da406ace0 msgr2=0x7f7da406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.609+0000 7f7dc77db700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7da406ace0 0x7f7da406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0151fc0 tx=0x7f7db0008040 comp rx=0 tx=0).stop 2023-06-08T03:51:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.609+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 msgr2=0x7f7dc00c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.609+0000 7f7dc77db700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 0x7f7dc00c5200 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7db800caa0 tx=0x7f7db800ce60 comp rx=0 tx=0).stop 2023-06-08T03:51:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 shutdown_connections 2023-06-08T03:51:27.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dc015c1f0 0x7f7dc00c5200 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7dc0071fe0 0x7f7dc0150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7da406ace0 0x7f7da406d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 --2- 172.21.15.79:0/2454085405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7dc00cf130 0x7f7dc0151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 >> 172.21.15.79:0/2454085405 conn(0x7f7dc00b1d30 msgr2=0x7f7dc00cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 shutdown_connections 2023-06-08T03:51:27.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.610+0000 7f7dc77db700 1 -- 172.21.15.79:0/2454085405 wait complete. 2023-06-08T03:51:27.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:27.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.746+0000 7f1e5b762700 1 Processor -- start 2023-06-08T03:51:27.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.746+0000 7f1e5b762700 1 -- start start 2023-06-08T03:51:27.752 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e5b762700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1e5415fb10 0x7f1e54070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e5b762700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e54070bd0 con 0x7f1e540cb1e0 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e5b762700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540cad50 con 0x7f1e540cce80 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e5b762700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540ce570 con 0x7f1e5415fb10 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e59cff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1e540cb1e0 0x7f1e540cb660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60230/0 (socket says 172.21.15.79:60230) 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e59cff700 1 -- 172.21.15.79:0/3644631737 learned_addr learned my addr 172.21.15.79:0/3644631737 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e58cfd700 1 --2- 172.21.15.79:0/3644631737 >> v2:172.21.15.156:3300/0 conn(0x7f1e5415fb10 0x7f1e54070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.753 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743883278 0 0) 0x7f1e4c001490 con 0x7f1e540cb1e0 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e400035b0 con 0x7f1e540cb1e0 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3796640264 0 0) 0x7f1e400035b0 con 0x7f1e540cb1e0 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 >> v2:172.21.15.156:3300/0 conn(0x7f1e5415fb10 msgr2=0x7f1e54070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 --2- 172.21.15.79:0/3644631737 >> v2:172.21.15.156:3300/0 conn(0x7f1e5415fb10 0x7f1e54070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 >> v1:172.21.15.156:6789/0 conn(0x7f1e540cce80 legacy=0x7f1e540cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54160080 con 0x7f1e540cb1e0 2023-06-08T03:51:27.754 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f1e4c001b80 con 0x7f1e540cb1e0 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 >> v1:172.21.15.79:6789/0 conn(0x7f1e540cb1e0 legacy=0x7f1e540cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e40003a00 0x7f1e40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e40004850 0x7f1e40004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40005200 0x7f1e4000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540ce570 con 0x7f1e40003a00 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c080 con 0x7f1e40004850 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4000c240 con 0x7f1e40005200 2023-06-08T03:51:27.755 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.747+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e40003a00 0x7f1e40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e58cfd700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e40004850 0x7f1e40004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e59cff700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40005200 0x7f1e4000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e594fe700 1 -- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40005200 msgr2=0x7f1e4000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40005200 0x7f1e4000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e594fe700 1 -- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e40004850 msgr2=0x7f1e40004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e40004850 0x7f1e40004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.756 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e594fe700 1 -- 172.21.15.79:0/3644631737 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e400035b0 con 0x7f1e40003a00 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e58cfd700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e40004850 0x7f1e40004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e59cff700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40005200 0x7f1e4000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e59cff700 1 -- 172.21.15.79:0/3644631737 reap_dead start 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e40003a00 0x7f1e40003e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e500017c0 tx=0x7f1e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df07b6c1eac38f57 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500052d0 con 0x7f1e40003a00 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e50005430 con 0x7f1e40003a00 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.748+0000 7f1e4bfff700 1 -- 172.21.15.79:0/3644631737 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e500038c0 con 0x7f1e40003a00 2023-06-08T03:51:27.757 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 -- 172.21.15.79:0/3644631737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e40003a00 msgr2=0x7f1e40003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e40003a00 0x7f1e40003e10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e500017c0 tx=0x7f1e40005880 comp rx=0 tx=0).stop 2023-06-08T03:51:27.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 -- 172.21.15.79:0/3644631737 shutdown_connections 2023-06-08T03:51:27.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3644631737 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e40003a00 0x7f1e40003e10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 -- 172.21.15.79:0/3644631737 >> 172.21.15.79:0/3644631737 conn(0x7f1e5406e1e0 msgr2=0x7f1e5406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:27.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 -- 172.21.15.79:0/3644631737 shutdown_connections 2023-06-08T03:51:27.758 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.749+0000 7f1e5b762700 1 -- 172.21.15.79:0/3644631737 wait complete. 2023-06-08T03:51:27.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.750+0000 7f1e5b762700 1 Processor -- start 2023-06-08T03:51:27.760 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.750+0000 7f1e5b762700 1 -- start start 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40003a00 0x7f1e540d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e541f2f60 0x7f1e540c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 0x7f1e541f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c99f0 con 0x7f1e540c8e00 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b16d0 con 0x7f1e541f2f60 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540b1970 con 0x7f1e40003a00 2023-06-08T03:51:27.761 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 0x7f1e541f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 0x7f1e541f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36766/0 (socket says 172.21.15.79:36766) 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 -- 172.21.15.79:0/3449868194 learned_addr learned my addr 172.21.15.79:0/3449868194 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40003a00 0x7f1e540d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 -- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40003a00 msgr2=0x7f1e540d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40003a00 0x7f1e540d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 -- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e541f2f60 msgr2=0x7f1e540c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.762 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e58cfd700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e541f2f60 0x7f1e540c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e541f2f60 0x7f1e540c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 -- 172.21.15.79:0/3449868194 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e540c9430 con 0x7f1e540c8e00 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40003a00 0x7f1e540d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e58cfd700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e541f2f60 0x7f1e540c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e59cff700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 0x7f1e541f20f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e5415fee0 tx=0x7f1e4c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c00ecd0 con 0x7f1e540c8e00 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4c00ee30 con 0x7f1e540c8e00 2023-06-08T03:51:27.763 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540c9710 con 0x7f1e540c8e00 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.751+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c013620 con 0x7f1e540c8e00 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.752+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e540d1090 con 0x7f1e540c8e00 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.752+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e540d1220 con 0x7f1e540c8e00 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.753+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e4c013780 con 0x7f1e540c8e00 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.753+0000 7f1e49ffb700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e3806ace0 0x7f1e3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.753+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1e4c089b10 con 0x7f1e540c8e00 2023-06-08T03:51:27.764 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.753+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e3806ace0 0x7f1e3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:27.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.754+0000 7f1e594fe700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e3806ace0 0x7f1e3806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cd370 tx=0x7f1e50009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:27.765 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.758+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e4c0548d0 con 0x7f1e540c8e00 2023-06-08T03:51:27.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.945+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1e54070bd0 con 0x7f1e540c8e00 2023-06-08T03:51:27.949 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.945+0000 7f1e49ffb700 1 -- 172.21.15.79:0/3449868194 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1e54070bd0 con 0x7f1e540c8e00 2023-06-08T03:51:27.949 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:27.951 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e3806ace0 msgr2=0x7f1e3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e3806ace0 0x7f1e3806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e540cd370 tx=0x7f1e50009030 comp rx=0 tx=0).stop 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 msgr2=0x7f1e541f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 0x7f1e541f20f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1e5415fee0 tx=0x7f1e4c00e440 comp rx=0 tx=0).stop 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 shutdown_connections 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e540c8e00 0x7f1e541f20f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e3806ace0 0x7f1e3806d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e40003a00 0x7f1e540d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 --2- 172.21.15.79:0/3449868194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e541f2f60 0x7f1e540c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:27.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.947+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 >> 172.21.15.79:0/3449868194 conn(0x7f1e5406e1e0 msgr2=0x7f1e540b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:27.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.948+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 shutdown_connections 2023-06-08T03:51:27.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:27.948+0000 7f1e5b762700 1 -- 172.21.15.79:0/3449868194 wait complete. 2023-06-08T03:51:27.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:28.084 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 Processor -- start 2023-06-08T03:51:28.085 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 -- start start 2023-06-08T03:51:28.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd1015ffa0 0x7fdd10160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd100b29a0 0x7fdd101608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.086 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdd1006c750 0x7fdd100cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1006ae60 con 0x7fdd1015ffa0 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1015d430 con 0x7fdd1006c750 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.081+0000 7fdd1549f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1015d710 con 0x7fdd100b29a0 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd1015ffa0 0x7fdd10160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd1015ffa0 0x7fdd10160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36770/0 (socket says 172.21.15.79:36770) 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 -- 172.21.15.79:0/2401737559 learned_addr learned my addr 172.21.15.79:0/2401737559 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:28.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0e7fc700 1 --2- 172.21.15.79:0/2401737559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd100b29a0 0x7fdd101608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0f7fe700 1 --2- 172.21.15.79:0/2401737559 >> v2:172.21.15.156:3300/0 conn(0x7fdd1006c750 0x7fdd100cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 -- 172.21.15.79:0/2401737559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd100b29a0 msgr2=0x7fdd101608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 --2- 172.21.15.79:0/2401737559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd100b29a0 0x7fdd101608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 -- 172.21.15.79:0/2401737559 >> v2:172.21.15.156:3300/0 conn(0x7fdd1006c750 msgr2=0x7fdd100cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 --2- 172.21.15.79:0/2401737559 >> v2:172.21.15.156:3300/0 conn(0x7fdd1006c750 0x7fdd100cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 -- 172.21.15.79:0/2401737559 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100ce5c0 con 0x7fdd1015ffa0 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0effd700 1 --2- 172.21.15.79:0/2401737559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd10160370 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdd100d1740 tx=0x7fdd00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6106e3e6966c75 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0dffb700 1 -- 172.21.15.79:0/2401737559 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00011830 con 0x7fdd1015ffa0 2023-06-08T03:51:28.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0dffb700 1 -- 172.21.15.79:0/2401737559 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd00011990 con 0x7fdd1015ffa0 2023-06-08T03:51:28.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.082+0000 7fdd0dffb700 1 -- 172.21.15.79:0/2401737559 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00011d20 con 0x7fdd1015ffa0 2023-06-08T03:51:28.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 -- 172.21.15.79:0/2401737559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 msgr2=0x7fdd10160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 --2- 172.21.15.79:0/2401737559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd10160370 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdd100d1740 tx=0x7fdd00009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 -- 172.21.15.79:0/2401737559 shutdown_connections 2023-06-08T03:51:28.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 --2- 172.21.15.79:0/2401737559 >> v2:172.21.15.156:3300/0 conn(0x7fdd1006c750 0x7fdd100cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 --2- 172.21.15.79:0/2401737559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd10160370 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.091 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 --2- 172.21.15.79:0/2401737559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd100b29a0 0x7fdd101608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 -- 172.21.15.79:0/2401737559 >> 172.21.15.79:0/2401737559 conn(0x7fdd10069d50 msgr2=0x7fdd1006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 -- 172.21.15.79:0/2401737559 shutdown_connections 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.083+0000 7fdd1549f700 1 -- 172.21.15.79:0/2401737559 wait complete. 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.084+0000 7fdd1549f700 1 Processor -- start 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.084+0000 7fdd1549f700 1 -- start start 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd1549f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1006c750 0x7fdd101561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.092 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd1549f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd101e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd1549f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd101e7f80 0x7fdd101e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd1549f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100b5a50 con 0x7fdd1015ffa0 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd1549f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd101e69a0 con 0x7fdd1006c750 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd1549f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd101e6c80 con 0x7fdd101e7f80 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd101e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd101e7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36778/0 (socket says 172.21.15.79:36778) 2023-06-08T03:51:28.093 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 -- 172.21.15.79:0/357358098 learned_addr learned my addr 172.21.15.79:0/357358098 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0effd700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1006c750 0x7fdd101561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0f7fe700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd101e7f80 0x7fdd101e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 -- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd101e7f80 msgr2=0x7fdd101e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd101e7f80 0x7fdd101e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 -- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1006c750 msgr2=0x7fdd101561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1006c750 0x7fdd101561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 -- 172.21.15.79:0/357358098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00009550 con 0x7fdd1015ffa0 2023-06-08T03:51:28.094 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.085+0000 7fdd0e7fc700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd101e7a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e9b0 tx=0x7fdd0400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.086+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04009710 con 0x7fdd1015ffa0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.086+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0400ee50 con 0x7fdd1015ffa0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.086+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd101e6f60 con 0x7fdd1015ffa0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.086+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400ca50 con 0x7fdd1015ffa0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.086+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd101e7520 con 0x7fdd1015ffa0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.086+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd101e76e0 con 0x7fdd1015ffa0 2023-06-08T03:51:28.095 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.087+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd0400cbb0 con 0x7fdd1015ffa0 2023-06-08T03:51:28.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.087+0000 7fdcf77fe700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdcfc06ace0 0x7fdcfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.087+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdd040896c0 con 0x7fdd1015ffa0 2023-06-08T03:51:28.096 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.088+0000 7fdd0effd700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdcfc06ace0 0x7fdcfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.088+0000 7fdd0effd700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdcfc06ace0 0x7fdcfc06d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd00014040 tx=0x7fdd00008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.097 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.092+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd04057fa0 con 0x7fdd1015ffa0 2023-06-08T03:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.297+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdd10073e90 con 0x7fdd1015ffa0 2023-06-08T03:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.297+0000 7fdcf77fe700 1 -- 172.21.15.79:0/357358098 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdd0400ce60 con 0x7fdd1015ffa0 2023-06-08T03:51:28.303 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdcfc06ace0 msgr2=0x7fdcfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdcfc06ace0 0x7fdcfc06d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdd00014040 tx=0x7fdd00008030 comp rx=0 tx=0).stop 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 msgr2=0x7fdd101e7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd101e7a40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e9b0 tx=0x7fdd0400ecc0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 shutdown_connections 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdcfc06ace0 0x7fdcfc06d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd101e7f80 0x7fdd101e5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1006c750 0x7fdd101561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 --2- 172.21.15.79:0/357358098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd1015ffa0 0x7fdd101e7a40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 >> 172.21.15.79:0/357358098 conn(0x7fdd10069d50 msgr2=0x7fdd1006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 shutdown_connections 2023-06-08T03:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.299+0000 7fdd1549f700 1 -- 172.21.15.79:0/357358098 wait complete. 2023-06-08T03:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:28.315 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2023-06-08T03:51:28.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:28.316 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-06-08T03:51:28.351 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:28.352 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T03:51:28.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:28.352 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-08T03:51:28.408 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:51:28.409 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:28.430 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:51:28.430 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:28.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:28.610 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.606+0000 7f42a41fb700 1 Processor -- start 2023-06-08T03:51:28.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.607+0000 7f42a41fb700 1 -- start start 2023-06-08T03:51:28.611 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.607+0000 7f42a41fb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f429c15aa10 0x7f429c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.607+0000 7f42a41fb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 0x7f429c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.607+0000 7f42a41fb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c0b0750 con 0x7f429c15a640 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.607+0000 7f42a41fb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f429c0aff60 con 0x7f429c15aa10 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.607+0000 7f42a41fb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0ceeb0 con 0x7f429c0af580 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 0x7f429c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a1796700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f429c15aa10 0x7f429c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.612 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 0x7f429c0afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:51110/0 (socket says 172.21.15.79:51110) 2023-06-08T03:51:28.613 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 -- 172.21.15.79:0/3726991362 learned_addr learned my addr 172.21.15.79:0/3726991362 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:28.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 -- 172.21.15.79:0/3726991362 >> v2:172.21.15.156:3300/0 conn(0x7f429c15aa10 msgr2=0x7f429c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:28.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 --2- 172.21.15.79:0/3726991362 >> v2:172.21.15.156:3300/0 conn(0x7f429c15aa10 0x7f429c0b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 -- 172.21.15.79:0/3726991362 >> v1:172.21.15.156:6789/0 conn(0x7f429c15a640 legacy=0x7f429c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.614 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 -- 172.21.15.79:0/3726991362 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c0b09a0 con 0x7f429c0af580 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a1796700 1 --2- 172.21.15.79:0/3726991362 >> v2:172.21.15.156:3300/0 conn(0x7f429c15aa10 0x7f429c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a2798700 1 --2- 172.21.15.79:0/3726991362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 0x7f429c0afa20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4294009e50 tx=0x7f429400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45128314f06ef085 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a0f95700 1 -- 172.21.15.79:0/3726991362 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400e070 con 0x7f429c0af580 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a0f95700 1 -- 172.21.15.79:0/3726991362 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400b910 con 0x7f429c0af580 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.608+0000 7f42a0f95700 1 -- 172.21.15.79:0/3726991362 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400bcc0 con 0x7f429c0af580 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 -- 172.21.15.79:0/3726991362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 msgr2=0x7f429c0afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.615 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 --2- 172.21.15.79:0/3726991362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 0x7f429c0afa20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4294009e50 tx=0x7f429400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 -- 172.21.15.79:0/3726991362 shutdown_connections 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 --2- 172.21.15.79:0/3726991362 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c0af580 0x7f429c0afa20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 --2- 172.21.15.79:0/3726991362 >> v2:172.21.15.156:3300/0 conn(0x7f429c15aa10 0x7f429c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 -- 172.21.15.79:0/3726991362 >> 172.21.15.79:0/3726991362 conn(0x7f429c06a670 msgr2=0x7f429c099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 -- 172.21.15.79:0/3726991362 shutdown_connections 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.609+0000 7f42a41fb700 1 -- 172.21.15.79:0/3726991362 wait complete. 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.610+0000 7f42a41fb700 1 Processor -- start 2023-06-08T03:51:28.616 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.610+0000 7f42a41fb700 1 -- start start 2023-06-08T03:51:28.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a41fb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0af580 0x7f429c156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a41fb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 0x7f429c0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a41fb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c15aa10 0x7f429c159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a41fb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c156880 con 0x7f429c15a640 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a41fb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0b54a0 con 0x7f429c0af580 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a41fb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0b5930 con 0x7f429c15aa10 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 0x7f429c0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 0x7f429c0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36792/0 (socket says 172.21.15.79:36792) 2023-06-08T03:51:28.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 -- 172.21.15.79:0/2842497860 learned_addr learned my addr 172.21.15.79:0/2842497860 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1f97700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0af580 0x7f429c156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a2798700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c15aa10 0x7f429c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 -- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c15aa10 msgr2=0x7f429c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c15aa10 0x7f429c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 -- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0af580 msgr2=0x7f429c156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0af580 0x7f429c156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.611+0000 7f42a1796700 1 -- 172.21.15.79:0/2842497860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42980094a0 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.612+0000 7f42a1796700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 0x7f429c0b4060 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f429800ebe0 tx=0x7f429800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.612+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298009760 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.612+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4294009b30 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.612+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429800c850 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.612+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800cbe0 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.612+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c0b4b70 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.613+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c0d08a0 con 0x7f429c15a640 2023-06-08T03:51:28.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.613+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f429800cd40 con 0x7f429c15a640 2023-06-08T03:51:28.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.614+0000 7f42927fc700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f428006ace0 0x7f428006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.614+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f42980898a0 con 0x7f429c15a640 2023-06-08T03:51:28.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.614+0000 7f42a1f97700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f428006ace0 0x7f428006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.614+0000 7f42a1f97700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f428006ace0 0x7f428006d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f428c0018a0 tx=0x7f428c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.618+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4298054540 con 0x7f429c15a640 2023-06-08T03:51:28.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.809+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f429c0b5610 con 0x7f429c15a640 2023-06-08T03:51:28.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.811+0000 7f42927fc700 1 -- 172.21.15.79:0/2842497860 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f4298057b60 con 0x7f429c15a640 2023-06-08T03:51:28.815 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:28.815 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:28.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f428006ace0 msgr2=0x7f428006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f428006ace0 0x7f428006d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f428c0018a0 tx=0x7f428c009040 comp rx=0 tx=0).stop 2023-06-08T03:51:28.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 msgr2=0x7f429c0b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 0x7f429c0b4060 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f429800ebe0 tx=0x7f429800eef0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 shutdown_connections 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f428006ace0 0x7f428006d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f429c15aa10 0x7f429c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f429c0af580 0x7f429c156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 --2- 172.21.15.79:0/2842497860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429c15a640 0x7f429c0b4060 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 >> 172.21.15.79:0/2842497860 conn(0x7f429c06a670 msgr2=0x7f429c06e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 shutdown_connections 2023-06-08T03:51:28.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.813+0000 7f42a41fb700 1 -- 172.21.15.79:0/2842497860 wait complete. 2023-06-08T03:51:28.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:28.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:28.952 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.948+0000 7f7a9cf2c700 1 Processor -- start 2023-06-08T03:51:28.953 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.948+0000 7f7a9cf2c700 1 -- start start 2023-06-08T03:51:28.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.949+0000 7f7a9cf2c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a980d0060 0x7f7a980af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.949+0000 7f7a9cf2c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980cae50 con 0x7f7a980cd080 2023-06-08T03:51:28.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.949+0000 7f7a9cf2c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980b2f10 con 0x7f7a980cf490 2023-06-08T03:51:28.956 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.949+0000 7f7a9cf2c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a980cbea0 con 0x7f7a980d0060 2023-06-08T03:51:28.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.949+0000 7f7a95d9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7a980cd080 0x7f7a980b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60274/0 (socket says 172.21.15.79:60274) 2023-06-08T03:51:28.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.949+0000 7f7a95d9b700 1 -- 172.21.15.79:0/1243093655 learned_addr learned my addr 172.21.15.79:0/1243093655 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:28.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701843872 0 0) 0x7f7a980cae50 con 0x7f7a980cd080 2023-06-08T03:51:28.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a800035b0 con 0x7f7a980cd080 2023-06-08T03:51:28.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9659c700 1 --2- 172.21.15.79:0/1243093655 >> v2:172.21.15.156:3300/0 conn(0x7f7a980d0060 0x7f7a980af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695085625 0 0) 0x7f7a800035b0 con 0x7f7a980cd080 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 >> v2:172.21.15.156:3300/0 conn(0x7f7a980d0060 msgr2=0x7f7a980af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 --2- 172.21.15.79:0/1243093655 >> v2:172.21.15.156:3300/0 conn(0x7f7a980d0060 0x7f7a980af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 >> v1:172.21.15.156:6789/0 conn(0x7f7a980cf490 legacy=0x7f7a980b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9815d930 con 0x7f7a980cd080 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f7a88001670 con 0x7f7a980cd080 2023-06-08T03:51:28.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 >> v1:172.21.15.79:6789/0 conn(0x7f7a980cd080 legacy=0x7f7a980b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a80003a50 0x7f7a80003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80004850 0x7f7a80004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a80005200 0x7f7a8000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000c0e0 con 0x7f7a80005200 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000c220 con 0x7f7a80004850 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000c3e0 con 0x7f7a80003a50 2023-06-08T03:51:28.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a96d9d700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a80005200 0x7f7a8000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a95d9b700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80004850 0x7f7a80004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.950+0000 7f7a9659c700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a80003a50 0x7f7a80003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 -- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a80003a50 msgr2=0x7f7a80003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a80003a50 0x7f7a80003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 -- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80004850 msgr2=0x7f7a80004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:28.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80004850 0x7f7a80004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 -- 172.21.15.79:0/1243093655 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800035b0 con 0x7f7a80005200 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 -- 172.21.15.79:0/1243093655 reap_dead start 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a96d9d700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a80005200 0x7f7a8000ba10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c160 tx=0x7f7a80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889cf973d1088d8f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90001db0 con 0x7f7a80005200 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9000cca0 con 0x7f7a80005200 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9559a700 1 -- 172.21.15.79:0/1243093655 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90003c20 con 0x7f7a80005200 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/1243093655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a80005200 msgr2=0x7f7a8000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a80005200 0x7f7a8000ba10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000c160 tx=0x7f7a80005880 comp rx=0 tx=0).stop 2023-06-08T03:51:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/1243093655 shutdown_connections 2023-06-08T03:51:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/1243093655 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a80005200 0x7f7a8000ba10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/1243093655 >> 172.21.15.79:0/1243093655 conn(0x7f7a9806e1e0 msgr2=0x7f7a9806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/1243093655 shutdown_connections 2023-06-08T03:51:28.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.951+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/1243093655 wait complete. 2023-06-08T03:51:28.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.953+0000 7f7a9cf2c700 1 Processor -- start 2023-06-08T03:51:28.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.953+0000 7f7a9cf2c700 1 -- start start 2023-06-08T03:51:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.953+0000 7f7a9cf2c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80005200 0x7f7a98152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 0x7f7a980c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a980c5170 0x7f7a980c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c48e0 con 0x7f7a980c4d00 2023-06-08T03:51:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98152b70 con 0x7f7a80005200 2023-06-08T03:51:28.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98155e40 con 0x7f7a980c5170 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 0x7f7a980c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 0x7f7a980c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36810/0 (socket says 172.21.15.79:36810) 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 -- 172.21.15.79:0/758945859 learned_addr learned my addr 172.21.15.79:0/758945859 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9659c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80005200 0x7f7a98152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a96d9d700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a980c5170 0x7f7a980c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 -- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a980c5170 msgr2=0x7f7a980c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a980c5170 0x7f7a980c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 -- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80005200 msgr2=0x7f7a98152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80005200 0x7f7a98152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 -- 172.21.15.79:0/758945859 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980c69a0 con 0x7f7a980c4d00 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9659c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80005200 0x7f7a98152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a95d9b700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 0x7f7a980c9440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a880017a0 tx=0x7f7a88007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011580 con 0x7f7a980c4d00 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a880116e0 con 0x7f7a980c4d00 2023-06-08T03:51:28.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a980c4220 con 0x7f7a980c4d00 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011a90 con 0x7f7a980c4d00 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.954+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a980c6c80 con 0x7f7a980c4d00 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.955+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a98066f00 con 0x7f7a980c4d00 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.956+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a88011bf0 con 0x7f7a980c4d00 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.956+0000 7f7a86ffd700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a7806ace0 0x7f7a7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.956+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a88089770 con 0x7f7a980c4d00 2023-06-08T03:51:28.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.956+0000 7f7a9659c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a7806ace0 0x7f7a7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.957+0000 7f7a9659c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a7806ace0 0x7f7a7806d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009ae0 tx=0x7f7a8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:28.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:28.961+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7a880545c0 con 0x7f7a980c4d00 2023-06-08T03:51:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.144+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7a980c4a20 con 0x7f7a980c4d00 2023-06-08T03:51:29.148 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.144+0000 7f7a86ffd700 1 -- 172.21.15.79:0/758945859 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7a980c4a20 con 0x7f7a980c4d00 2023-06-08T03:51:29.149 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a7806ace0 msgr2=0x7f7a7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a7806ace0 0x7f7a7806d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009ae0 tx=0x7f7a8c008040 comp rx=0 tx=0).stop 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 msgr2=0x7f7a980c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 0x7f7a980c9440 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a880017a0 tx=0x7f7a88007340 comp rx=0 tx=0).stop 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 shutdown_connections 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a7806ace0 0x7f7a7806d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a980c5170 0x7f7a980c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a80005200 0x7f7a98152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.146+0000 7f7a9cf2c700 1 --2- 172.21.15.79:0/758945859 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a980c4d00 0x7f7a980c9440 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.147+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 >> 172.21.15.79:0/758945859 conn(0x7f7a9806e1e0 msgr2=0x7f7a980cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.147+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 shutdown_connections 2023-06-08T03:51:29.152 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.147+0000 7f7a9cf2c700 1 -- 172.21.15.79:0/758945859 wait complete. 2023-06-08T03:51:29.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:29.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.280+0000 7fdf7a3d7700 1 Processor -- start 2023-06-08T03:51:29.284 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.280+0000 7fdf7a3d7700 1 -- start start 2023-06-08T03:51:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf7a3d7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 0x7fdf74071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.287 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf7a3d7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdf74069d50 0x7fdf7406a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf7a3d7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf740cd140 con 0x7fdf7415c1f0 2023-06-08T03:51:29.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf7a3d7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7406a730 con 0x7fdf74069d50 2023-06-08T03:51:29.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf7a3d7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74071110 con 0x7fdf7406d8a0 2023-06-08T03:51:29.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf73fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdf7415c1f0 0x7fdf740cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60292/0 (socket says 172.21.15.79:60292) 2023-06-08T03:51:29.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf73fff700 1 -- 172.21.15.79:0/1200474918 learned_addr learned my addr 172.21.15.79:0/1200474918 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf737fe700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 0x7fdf74071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf78974700 1 --2- 172.21.15.79:0/1200474918 >> v2:172.21.15.156:3300/0 conn(0x7fdf74069d50 0x7fdf7406a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605912814 0 0) 0x7fdf740cd140 con 0x7fdf7415c1f0 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf5c0035b0 con 0x7fdf7415c1f0 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181358973 0 0) 0x7fdf5c0035b0 con 0x7fdf7415c1f0 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 msgr2=0x7fdf74071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 0x7fdf74071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 >> v2:172.21.15.156:3300/0 conn(0x7fdf74069d50 msgr2=0x7fdf7406a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 --2- 172.21.15.79:0/1200474918 >> v2:172.21.15.156:3300/0 conn(0x7fdf74069d50 0x7fdf7406a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74071be0 con 0x7fdf7415c1f0 2023-06-08T03:51:29.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fdf64001670 con 0x7fdf7415c1f0 2023-06-08T03:51:29.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 >> v1:172.21.15.79:6789/0 conn(0x7fdf7415c1f0 legacy=0x7fdf740cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 0x7fdf5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf5c004d90 0x7fdf5c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74071110 con 0x7fdf5c003a00 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c00ba10 con 0x7fdf5c004d90 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c00bbd0 con 0x7fdf7406d8a0 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.281+0000 7fdf73fff700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf737fe700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 0x7fdf5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf78974700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf5c004d90 0x7fdf5c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf73fff700 1 -- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 msgr2=0x7fdf5c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf73fff700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf7406d8a0 0x7fdf5c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf73fff700 1 -- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf5c004d90 msgr2=0x7fdf5c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf73fff700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf5c004d90 0x7fdf5c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf73fff700 1 -- 172.21.15.79:0/1200474918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0035b0 con 0x7fdf5c003a00 2023-06-08T03:51:29.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf78974700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf5c004d90 0x7fdf5c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf73fff700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400c370 tx=0x7fdf5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f8c4a1f0891ca3 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64007c60 con 0x7fdf5c003a00 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64005960 con 0x7fdf5c003a00 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf72ffd700 1 -- 172.21.15.79:0/1200474918 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64005d10 con 0x7fdf5c003a00 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/1200474918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf5c003a00 msgr2=0x7fdf5c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/1200474918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf5c003a00 0x7fdf5c003e10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400c370 tx=0x7fdf5c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf78974700 1 -- 172.21.15.79:0/1200474918 reap_dead start 2023-06-08T03:51:29.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/1200474918 shutdown_connections 2023-06-08T03:51:29.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.282+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/1200474918 >> 172.21.15.79:0/1200474918 conn(0x7fdf740b1d30 msgr2=0x7fdf7406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:29.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.283+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/1200474918 shutdown_connections 2023-06-08T03:51:29.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.283+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/1200474918 wait complete. 2023-06-08T03:51:29.294 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.284+0000 7fdf7a3d7700 1 Processor -- start 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.284+0000 7fdf7a3d7700 1 -- start start 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf5c003a00 0x7fdf740c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf741f4cc0 0x7fdf740cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 0x7fdf74149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740c30e0 con 0x7fdf74149580 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740c2db0 con 0x7fdf741f4cc0 2023-06-08T03:51:29.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74149f60 con 0x7fdf5c003a00 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 0x7fdf74149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 0x7fdf74149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36832/0 (socket says 172.21.15.79:36832) 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 -- 172.21.15.79:0/815135197 learned_addr learned my addr 172.21.15.79:0/815135197 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf73fff700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf5c003a00 0x7fdf740c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf737fe700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf741f4cc0 0x7fdf740cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 -- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf5c003a00 msgr2=0x7fdf740c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf5c003a00 0x7fdf740c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 -- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf741f4cc0 msgr2=0x7fdf740cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf741f4cc0 0x7fdf740cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 -- 172.21.15.79:0/815135197 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf741f5510 con 0x7fdf74149580 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf78974700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 0x7fdf74149a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdf740707a0 tx=0x7fdf6c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c01b070 con 0x7fdf74149580 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c00f040 con 0x7fdf74149580 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf741f57f0 con 0x7fdf74149580 2023-06-08T03:51:29.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00e050 con 0x7fdf74149580 2023-06-08T03:51:29.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.285+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf741f5130 con 0x7fdf74149580 2023-06-08T03:51:29.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.286+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf741f52c0 con 0x7fdf74149580 2023-06-08T03:51:29.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.287+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf6c010d50 con 0x7fdf74149580 2023-06-08T03:51:29.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.287+0000 7fdf70ff9700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf6006ace0 0x7fdf6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.287+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf6c089760 con 0x7fdf74149580 2023-06-08T03:51:29.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.287+0000 7fdf73fff700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf6006ace0 0x7fdf6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.288+0000 7fdf73fff700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf6006ace0 0x7fdf6006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf64001930 tx=0x7fdf6400a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.301 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.291+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf6c054520 con 0x7fdf74149580 2023-06-08T03:51:29.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.477+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf74073e90 con 0x7fdf74149580 2023-06-08T03:51:29.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.478+0000 7fdf70ff9700 1 -- 172.21.15.79:0/815135197 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf6c057b40 con 0x7fdf74149580 2023-06-08T03:51:29.482 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf6006ace0 msgr2=0x7fdf6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf6006ace0 0x7fdf6006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf64001930 tx=0x7fdf6400a000 comp rx=0 tx=0).stop 2023-06-08T03:51:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 msgr2=0x7fdf74149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 0x7fdf74149a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdf740707a0 tx=0x7fdf6c0104a0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 shutdown_connections 2023-06-08T03:51:29.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdf74149580 0x7fdf74149a20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdf6006ace0 0x7fdf6006d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdf5c003a00 0x7fdf740c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 --2- 172.21.15.79:0/815135197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf741f4cc0 0x7fdf740cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 >> 172.21.15.79:0/815135197 conn(0x7fdf740b1d30 msgr2=0x7fdf740c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 shutdown_connections 2023-06-08T03:51:29.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.480+0000 7fdf7a3d7700 1 -- 172.21.15.79:0/815135197 wait complete. 2023-06-08T03:51:29.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:29.617 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.613+0000 7f5cbe27c700 1 Processor -- start 2023-06-08T03:51:29.618 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cbe27c700 1 -- start start 2023-06-08T03:51:29.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cbe27c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 0x7f5cb80ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cbe27c700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb80cd7b0 con 0x7f5cb815d3d0 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cbe27c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb80cda50 con 0x7f5cb80cc690 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cbe27c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb80cdd30 con 0x7f5cb8158d50 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cb77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5cb815d3d0 0x7f5cb80cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60312/0 (socket says 172.21.15.79:60312) 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.614+0000 7f5cb77fe700 1 -- 172.21.15.79:0/234346169 learned_addr learned my addr 172.21.15.79:0/234346169 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb7fff700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 0x7f5cb80ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274863008 0 0) 0x7f5cb80cd7b0 con 0x7f5cb815d3d0 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca00035b0 con 0x7f5cb815d3d0 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 377140990 0 0) 0x7f5ca00035b0 con 0x7f5cb815d3d0 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 msgr2=0x7f5cb80ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 0x7f5cb80ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 >> v1:172.21.15.156:6789/0 conn(0x7f5cb80cc690 legacy=0x7f5cb8071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80ce010 con 0x7f5cb815d3d0 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f5ca8001670 con 0x7f5cb815d3d0 2023-06-08T03:51:29.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 >> v1:172.21.15.79:6789/0 conn(0x7f5cb815d3d0 legacy=0x7f5cb80cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 0x7f5ca0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ca0003f70 0x7f5ca0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0004d90 0x7f5ca0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb80cdd30 con 0x7f5ca0004d90 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca000ba40 con 0x7f5ca0003f70 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca000bc00 con 0x7f5cb8158d50 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cbc819700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0004d90 0x7f5ca0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb77fe700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ca0003f70 0x7f5ca0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cb7fff700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 0x7f5ca0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cbc819700 1 -- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 msgr2=0x7f5ca0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cbc819700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb8158d50 0x7f5ca0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cbc819700 1 -- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ca0003f70 msgr2=0x7f5ca0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cbc819700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ca0003f70 0x7f5ca0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.615+0000 7f5cbc819700 1 -- 172.21.15.79:0/234346169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca00035b0 con 0x7f5ca0004d90 2023-06-08T03:51:29.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbc819700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0004d90 0x7f5ca0005200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c260 tx=0x7f5ca0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9235d213d1b917 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0001e70 con 0x7f5ca0004d90 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb000ce20 con 0x7f5ca0004d90 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cb6ffd700 1 -- 172.21.15.79:0/234346169 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0012660 con 0x7f5ca0004d90 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbe27c700 1 -- 172.21.15.79:0/234346169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0004d90 msgr2=0x7f5ca0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/234346169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca0004d90 0x7f5ca0005200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c260 tx=0x7f5ca0005880 comp rx=0 tx=0).stop 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbc819700 1 -- 172.21.15.79:0/234346169 reap_dead start 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbe27c700 1 -- 172.21.15.79:0/234346169 shutdown_connections 2023-06-08T03:51:29.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbe27c700 1 -- 172.21.15.79:0/234346169 >> 172.21.15.79:0/234346169 conn(0x7f5cb80b1d30 msgr2=0x7f5cb806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:29.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbe27c700 1 -- 172.21.15.79:0/234346169 shutdown_connections 2023-06-08T03:51:29.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.616+0000 7f5cbe27c700 1 -- 172.21.15.79:0/234346169 wait complete. 2023-06-08T03:51:29.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 Processor -- start 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 -- start start 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 0x7f5cb81f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb81f1e80 0x7f5cb80ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb81f22f0 0x7f5cb80ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb80abd60 con 0x7f5cb81f58d0 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81f2d70 con 0x7f5cb81f1e80 2023-06-08T03:51:29.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.618+0000 7f5cbe27c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81f27b0 con 0x7f5cb81f22f0 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 0x7f5cb81f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 0x7f5cb81f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36848/0 (socket says 172.21.15.79:36848) 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 -- 172.21.15.79:0/3767524051 learned_addr learned my addr 172.21.15.79:0/3767524051 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb77fe700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb81f1e80 0x7f5cb80ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cbc819700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb81f22f0 0x7f5cb80ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 -- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb81f22f0 msgr2=0x7f5cb80ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb81f22f0 0x7f5cb80ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 -- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb81f1e80 msgr2=0x7f5cb80ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb81f1e80 0x7f5cb80ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 -- 172.21.15.79:0/3767524051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb81f2a90 con 0x7f5cb81f58d0 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb7fff700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 0x7f5cb81f5ca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5cac011aa0 tx=0x7f5cac011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00f980 con 0x7f5cb81f58d0 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac00fae0 con 0x7f5cb81f58d0 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb80ab9f0 con 0x7f5cb81f58d0 2023-06-08T03:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00fe90 con 0x7f5cb81f58d0 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.619+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb80af870 con 0x7f5cb81f58d0 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.620+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb80ac040 con 0x7f5cb81f58d0 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.620+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5cac014440 con 0x7f5cb81f58d0 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.621+0000 7f5cb4ff9700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5ca406ace0 0x7f5ca406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.621+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5cac089ed0 con 0x7f5cb81f58d0 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.621+0000 7f5cb77fe700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5ca406ace0 0x7f5ca406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.622+0000 7f5cb77fe700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5ca406ace0 0x7f5ca406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8001050 tx=0x7f5ca8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.625+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5cac058870 con 0x7f5cb81f58d0 2023-06-08T03:51:29.817 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.814+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cb8073e90 con 0x7f5cb81f58d0 2023-06-08T03:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.814+0000 7f5cb4ff9700 1 -- 172.21.15.79:0/3767524051 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cac058400 con 0x7f5cb81f58d0 2023-06-08T03:51:29.818 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5ca406ace0 msgr2=0x7f5ca406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.820 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5ca406ace0 0x7f5ca406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8001050 tx=0x7f5ca8009040 comp rx=0 tx=0).stop 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 msgr2=0x7f5cb81f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 0x7f5cb81f5ca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5cac011aa0 tx=0x7f5cac011e60 comp rx=0 tx=0).stop 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 shutdown_connections 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5ca406ace0 0x7f5ca406d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cb81f22f0 0x7f5cb80ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb81f58d0 0x7f5cb81f5ca0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.821 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 --2- 172.21.15.79:0/3767524051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb81f1e80 0x7f5cb80ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 >> 172.21.15.79:0/3767524051 conn(0x7f5cb80b1d30 msgr2=0x7f5cb80aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 shutdown_connections 2023-06-08T03:51:29.822 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.816+0000 7f5cbe27c700 1 -- 172.21.15.79:0/3767524051 wait complete. 2023-06-08T03:51:29.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:29.954 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.950+0000 7fe4f88b3700 1 Processor -- start 2023-06-08T03:51:29.954 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.950+0000 7fe4f88b3700 1 -- start start 2023-06-08T03:51:29.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f88b3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe4f406d8a0 0x7fe4f4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f88b3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 0x7fe4f406a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f88b3700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40cd140 con 0x7fe4f415c1f0 2023-06-08T03:51:29.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f88b3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f406a730 con 0x7fe4f406d8a0 2023-06-08T03:51:29.957 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f88b3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f4071110 con 0x7fe4f4069d50 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f259c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe4f415c1f0 0x7fe4f40cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60332/0 (socket says 172.21.15.79:60332) 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f259c700 1 -- 172.21.15.79:0/3247765571 learned_addr learned my addr 172.21.15.79:0/3247765571 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969501443 0 0) 0x7fe4f40cd140 con 0x7fe4f415c1f0 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/3247765571 >> v2:172.21.15.156:3300/0 conn(0x7fe4f406d8a0 0x7fe4f4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f2d9d700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 0x7fe4f406a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.951+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4dc0035b0 con 0x7fe4f415c1f0 2023-06-08T03:51:29.958 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650266922 0 0) 0x7fe4dc0035b0 con 0x7fe4f415c1f0 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 msgr2=0x7fe4f406a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 0x7fe4f406a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 >> v2:172.21.15.156:3300/0 conn(0x7fe4f406d8a0 msgr2=0x7fe4f4071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 --2- 172.21.15.79:0/3247765571 >> v2:172.21.15.156:3300/0 conn(0x7fe4f406d8a0 0x7fe4f4071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4071be0 con 0x7fe4f415c1f0 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fe4e8001670 con 0x7fe4f415c1f0 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 >> v1:172.21.15.79:6789/0 conn(0x7fe4f415c1f0 legacy=0x7fe4f40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.959 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc003a00 0x7fe4dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4dc004850 0x7fe4dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 0x7fe4dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f4071110 con 0x7fe4dc004850 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc00ba40 con 0x7fe4dc003a00 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc00bc00 con 0x7fe4f4069d50 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4dc004850 0x7fe4dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.960 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f259c700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc003a00 0x7fe4dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f2d9d700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 0x7fe4dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 msgr2=0x7fe4dc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f4069d50 0x7fe4dc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc003a00 msgr2=0x7fe4dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc003a00 0x7fe4dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/3247765571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0035b0 con 0x7fe4dc004850 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4dc004850 0x7fe4dc004cc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009b60 tx=0x7fe4dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43446c04f50f140b server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.961 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400e6c0 con 0x7fe4dc004850 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e400e820 con 0x7fe4dc004850 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.952+0000 7fe4f159a700 1 -- 172.21.15.79:0/3247765571 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400eb90 con 0x7fe4dc004850 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f88b3700 1 -- 172.21.15.79:0/3247765571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4dc004850 msgr2=0x7fe4dc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/3247765571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4dc004850 0x7fe4dc004cc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009b60 tx=0x7fe4dc005880 comp rx=0 tx=0).stop 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f2d9d700 1 -- 172.21.15.79:0/3247765571 reap_dead start 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f88b3700 1 -- 172.21.15.79:0/3247765571 shutdown_connections 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f88b3700 1 -- 172.21.15.79:0/3247765571 >> 172.21.15.79:0/3247765571 conn(0x7fe4f40b1d30 msgr2=0x7fe4f406d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:29.962 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f88b3700 1 -- 172.21.15.79:0/3247765571 shutdown_connections 2023-06-08T03:51:29.963 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.953+0000 7fe4f88b3700 1 -- 172.21.15.79:0/3247765571 wait complete. 2023-06-08T03:51:29.964 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.954+0000 7fe4f88b3700 1 Processor -- start 2023-06-08T03:51:29.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.954+0000 7fe4f88b3700 1 -- start start 2023-06-08T03:51:29.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f88b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc004850 0x7fe4f4148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f88b3700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 0x7fe4f415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f88b3700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f415a170 0x7fe4f415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f88b3700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f415e340 con 0x7fe4f415d990 2023-06-08T03:51:29.965 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f88b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f415ab20 con 0x7fe4dc004850 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f88b3700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40c6990 con 0x7fe4f415a170 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 0x7fe4f415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 0x7fe4f415de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36870/0 (socket says 172.21.15.79:36870) 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/2910508708 learned_addr learned my addr 172.21.15.79:0/2910508708 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f2d9d700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f415a170 0x7fe4f415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f259c700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc004850 0x7fe4f4148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f415a170 msgr2=0x7fe4f415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.966 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f415a170 0x7fe4f415a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc004850 msgr2=0x7fe4f4148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc004850 0x7fe4f4148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.955+0000 7fe4f1d9b700 1 -- 172.21.15.79:0/2910508708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f415f160 con 0x7fe4f415d990 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.956+0000 7fe4f1d9b700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 0x7fe4f415de00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40042f0 tx=0x7fe4e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.956+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400e6c0 con 0x7fe4f415d990 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.956+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e40038b0 con 0x7fe4f415d990 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.956+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f4148fc0 con 0x7fe4f415d990 2023-06-08T03:51:29.967 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.956+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4003c60 con 0x7fe4f415d990 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.956+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f415e830 con 0x7fe4f415d990 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.957+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f415f2f0 con 0x7fe4f415d990 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.957+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4e4013470 con 0x7fe4f415d990 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.958+0000 7fe4e2ffd700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe4d406ace0 0x7fe4d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.958+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe4e408aee0 con 0x7fe4f415d990 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.958+0000 7fe4f259c700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe4d406ace0 0x7fe4d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:29.968 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.958+0000 7fe4f259c700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe4d406ace0 0x7fe4d406d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80011e0 tx=0x7fe4e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:29.969 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:29.962+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe4e4055db0 con 0x7fe4f415d990 2023-06-08T03:51:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.147+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4f415f4f0 con 0x7fe4f415d990 2023-06-08T03:51:30.151 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.147+0000 7fe4e2ffd700 1 -- 172.21.15.79:0/2910508708 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4e40593d0 con 0x7fe4f415d990 2023-06-08T03:51:30.152 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe4d406ace0 msgr2=0x7fe4d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe4d406ace0 0x7fe4d406d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80011e0 tx=0x7fe4e8009040 comp rx=0 tx=0).stop 2023-06-08T03:51:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 msgr2=0x7fe4f415de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 0x7fe4f415de00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40042f0 tx=0x7fe4e40047c0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.154 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 shutdown_connections 2023-06-08T03:51:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe4d406ace0 0x7fe4d406d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4f415a170 0x7fe4f415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4dc004850 0x7fe4f4148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 --2- 172.21.15.79:0/2910508708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4f415d990 0x7fe4f415de00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.155 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 >> 172.21.15.79:0/2910508708 conn(0x7fe4f40b1d30 msgr2=0x7fe4f415b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 shutdown_connections 2023-06-08T03:51:30.156 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.150+0000 7fe4f88b3700 1 -- 172.21.15.79:0/2910508708 wait complete. 2023-06-08T03:51:30.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:30.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.284+0000 7f4a2f22b700 1 Processor -- start 2023-06-08T03:51:30.288 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.284+0000 7f4a2f22b700 1 -- start start 2023-06-08T03:51:30.289 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2f22b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a280cd4f0 0x7f4a280ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2f22b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a280cae50 con 0x7f4a280d0060 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2f22b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a280afee0 con 0x7f4a280cd080 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2f22b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a280cbea0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2d7c8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a280cd4f0 0x7f4a280ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2cfc7700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4a280d0060 0x7f4a280af7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60352/0 (socket says 172.21.15.79:60352) 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2d7c8700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a280cd4f0 0x7f4a280ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36880/0 (socket says 172.21.15.79:36880) 2023-06-08T03:51:30.290 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2cfc7700 1 -- 172.21.15.79:0/4050861190 learned_addr learned my addr 172.21.15.79:0/4050861190 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2d7c8700 1 -- 172.21.15.79:0/4050861190 >> v1:172.21.15.156:6789/0 conn(0x7f4a280cd080 legacy=0x7f4a280ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2d7c8700 1 -- 172.21.15.79:0/4050861190 >> v1:172.21.15.79:6789/0 conn(0x7f4a280d0060 legacy=0x7f4a280af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a2d7c8700 1 -- 172.21.15.79:0/4050861190 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2815d930 con 0x7f4a280cd4f0 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.285+0000 7f4a277fe700 1 -- 172.21.15.79:0/4050861190 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502280832 0 0) 0x7f4a280cae50 con 0x7f4a280d0060 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2d7c8700 1 --2- 172.21.15.79:0/4050861190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280ceb90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009d90 tx=0x7f4a2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff06bb6429eef43 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a277fe700 1 -- 172.21.15.79:0/4050861190 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000e070 con 0x7f4a280cd4f0 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a277fe700 1 -- 172.21.15.79:0/4050861190 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000bbd0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.291 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a277fe700 1 -- 172.21.15.79:0/4050861190 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20012430 con 0x7f4a280cd4f0 2023-06-08T03:51:30.292 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 -- 172.21.15.79:0/4050861190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 msgr2=0x7f4a280ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/4050861190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280ceb90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009d90 tx=0x7f4a2000b330 comp rx=0 tx=0).stop 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 -- 172.21.15.79:0/4050861190 shutdown_connections 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/4050861190 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280ceb90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 -- 172.21.15.79:0/4050861190 >> 172.21.15.79:0/4050861190 conn(0x7f4a2806e1e0 msgr2=0x7f4a2806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 -- 172.21.15.79:0/4050861190 shutdown_connections 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.286+0000 7f4a2f22b700 1 -- 172.21.15.79:0/4050861190 wait complete. 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.288+0000 7f4a2f22b700 1 Processor -- start 2023-06-08T03:51:30.293 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.288+0000 7f4a2f22b700 1 -- start start 2023-06-08T03:51:30.295 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2f22b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a280cd080 0x7f4a281550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2f22b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2f22b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a280d0060 0x7f4a281e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2f22b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280c5a80 con 0x7f4a280cd4f0 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2f22b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280c4b90 con 0x7f4a280cd080 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2f22b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280c4e70 con 0x7f4a280d0060 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.296 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36884/0 (socket says 172.21.15.79:36884) 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 -- 172.21.15.79:0/2798656276 learned_addr learned my addr 172.21.15.79:0/2798656276 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2cfc7700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a280cd080 0x7f4a281550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a2d7c8700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a280d0060 0x7f4a281e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 -- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a280d0060 msgr2=0x7f4a281e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a280d0060 0x7f4a281e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 -- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a280cd080 msgr2=0x7f4a281550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.297 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a280cd080 0x7f4a281550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.289+0000 7f4a27fff700 1 -- 172.21.15.79:0/2798656276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a20009a70 con 0x7f4a280cd4f0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.290+0000 7f4a27fff700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280c5f10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400c420 tx=0x7f4a1400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.290+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140052d0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.290+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14005430 con 0x7f4a280cd4f0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.290+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a280c5150 con 0x7f4a280cd4f0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.290+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140038c0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.290+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a280c5710 con 0x7f4a280cd4f0 2023-06-08T03:51:30.298 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.291+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a280c5bf0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.291+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a14003a20 con 0x7f4a280cd4f0 2023-06-08T03:51:30.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.291+0000 7f4a257fa700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4a0c06ace0 0x7f4a0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.292+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4a14088ba0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.292+0000 7f4a2cfc7700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4a0c06ace0 0x7f4a0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.299 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.292+0000 7f4a2cfc7700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4a0c06ace0 0x7f4a0c06d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c007b10 tx=0x7f4a1c007a60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.300 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.296+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4a140538d0 con 0x7f4a280cd4f0 2023-06-08T03:51:30.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.481+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4a28073e90 con 0x7f4a280cd4f0 2023-06-08T03:51:30.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.482+0000 7f4a257fa700 1 -- 172.21.15.79:0/2798656276 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4a14017020 con 0x7f4a280cd4f0 2023-06-08T03:51:30.486 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4a0c06ace0 msgr2=0x7f4a0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4a0c06ace0 0x7f4a0c06d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c007b10 tx=0x7f4a1c007a60 comp rx=0 tx=0).stop 2023-06-08T03:51:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 msgr2=0x7f4a280c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280c5f10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400c420 tx=0x7f4a1400c730 comp rx=0 tx=0).stop 2023-06-08T03:51:30.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 shutdown_connections 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4a0c06ace0 0x7f4a0c06d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a280d0060 0x7f4a281e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a280cd080 0x7f4a281550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 --2- 172.21.15.79:0/2798656276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a280cd4f0 0x7f4a280c5f10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 >> 172.21.15.79:0/2798656276 conn(0x7f4a2806e1e0 msgr2=0x7f4a2815c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 shutdown_connections 2023-06-08T03:51:30.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.484+0000 7f4a2f22b700 1 -- 172.21.15.79:0/2798656276 wait complete. 2023-06-08T03:51:30.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:30.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.617+0000 7f2665703700 1 Processor -- start 2023-06-08T03:51:30.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.617+0000 7f2665703700 1 -- start start 2023-06-08T03:51:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.617+0000 7f2665703700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26600cd700 0x7f266006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.617+0000 7f2665703700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266015fe90 con 0x7f266006ddd0 2023-06-08T03:51:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.617+0000 7f2665703700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2660160170 con 0x7f266015c0a0 2023-06-08T03:51:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.617+0000 7f2665703700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266006d4c0 con 0x7f26600cd700 2023-06-08T03:51:30.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265f7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f26600cd700 0x7f266006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265e7fc700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f266006ddd0 0x7f266015c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60368/0 (socket says 172.21.15.79:60368) 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265e7fc700 1 -- 172.21.15.79:0/674544339 learned_addr learned my addr 172.21.15.79:0/674544339 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265f7fe700 1 -- 172.21.15.79:0/674544339 >> v1:172.21.15.156:6789/0 conn(0x7f266015c0a0 legacy=0x7f266006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265f7fe700 1 -- 172.21.15.79:0/674544339 >> v1:172.21.15.79:6789/0 conn(0x7f266006ddd0 legacy=0x7f266015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265f7fe700 1 -- 172.21.15.79:0/674544339 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660160450 con 0x7f26600cd700 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265dffb700 1 -- 172.21.15.79:0/674544339 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567634042 0 0) 0x7f266015fe90 con 0x7f266006ddd0 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265f7fe700 1 --2- 172.21.15.79:0/674544339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26600cd700 0x7f266006cf80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2658009d90 tx=0x7f265800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95f63f665bddaaf server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265dffb700 1 -- 172.21.15.79:0/674544339 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800e070 con 0x7f26600cd700 2023-06-08T03:51:30.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265dffb700 1 -- 172.21.15.79:0/674544339 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265800bbd0 con 0x7f26600cd700 2023-06-08T03:51:30.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.618+0000 7f265dffb700 1 -- 172.21.15.79:0/674544339 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658012430 con 0x7f26600cd700 2023-06-08T03:51:30.624 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 -- 172.21.15.79:0/674544339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26600cd700 msgr2=0x7f266006cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 --2- 172.21.15.79:0/674544339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26600cd700 0x7f266006cf80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2658009d90 tx=0x7f265800b330 comp rx=0 tx=0).stop 2023-06-08T03:51:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 -- 172.21.15.79:0/674544339 shutdown_connections 2023-06-08T03:51:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 --2- 172.21.15.79:0/674544339 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f26600cd700 0x7f266006cf80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 -- 172.21.15.79:0/674544339 >> 172.21.15.79:0/674544339 conn(0x7f26600ced60 msgr2=0x7f26600cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 -- 172.21.15.79:0/674544339 shutdown_connections 2023-06-08T03:51:30.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.619+0000 7f2665703700 1 -- 172.21.15.79:0/674544339 wait complete. 2023-06-08T03:51:30.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.620+0000 7f2665703700 1 Processor -- start 2023-06-08T03:51:30.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 -- start start 2023-06-08T03:51:30.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 0x7f26600ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26600cd700 0x7f26600b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f266015c0a0 0x7f2660153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660154180 con 0x7f266006ddd0 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26601e8620 con 0x7f26600cd700 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f2665703700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26601e5d10 con 0x7f266015c0a0 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.621+0000 7f265effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 0x7f26600ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 0x7f26600ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36896/0 (socket says 172.21.15.79:36896) 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 -- 172.21.15.79:0/2348745981 learned_addr learned my addr 172.21.15.79:0/2348745981 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 -- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f266015c0a0 msgr2=0x7f2660153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265f7fe700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f266015c0a0 0x7f2660153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265e7fc700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26600cd700 0x7f26600b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f266015c0a0 0x7f2660153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 -- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26600cd700 msgr2=0x7f26600b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26600cd700 0x7f26600b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 -- 172.21.15.79:0/2348745981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658009a70 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f265effd700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 0x7f26600ae7c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f265000c450 tx=0x7f265000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500052d0 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650005430 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26601e59e0 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500038c0 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.622+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26600aee30 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.623+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26601e5840 con 0x7f266006ddd0 2023-06-08T03:51:30.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.623+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2650003a20 con 0x7f266006ddd0 2023-06-08T03:51:30.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.624+0000 7f26477fe700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f264c06ace0 0x7f264c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.624+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2650088b20 con 0x7f266006ddd0 2023-06-08T03:51:30.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.624+0000 7f265e7fc700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f264c06ace0 0x7f264c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.625+0000 7f265e7fc700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f264c06ace0 0x7f264c06d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2660159c60 tx=0x7f2654009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.628+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f26601e5840 con 0x7f266006ddd0 2023-06-08T03:51:30.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.832+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26601e5840 con 0x7f266006ddd0 2023-06-08T03:51:30.836 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.832+0000 7f26477fe700 1 -- 172.21.15.79:0/2348745981 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26601e5840 con 0x7f266006ddd0 2023-06-08T03:51:30.838 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f264c06ace0 msgr2=0x7f264c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f264c06ace0 0x7f264c06d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2660159c60 tx=0x7f2654009040 comp rx=0 tx=0).stop 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 msgr2=0x7f26600ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 0x7f26600ae7c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f265000c450 tx=0x7f265000c810 comp rx=0 tx=0).stop 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 shutdown_connections 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f264c06ace0 0x7f264c06d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f266015c0a0 0x7f2660153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.839 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f266006ddd0 0x7f26600ae7c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 --2- 172.21.15.79:0/2348745981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26600cd700 0x7f26600b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 >> 172.21.15.79:0/2348745981 conn(0x7f26600ced60 msgr2=0x7f266015ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:30.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 shutdown_connections 2023-06-08T03:51:30.840 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.834+0000 7f2665703700 1 -- 172.21.15.79:0/2348745981 wait complete. 2023-06-08T03:51:30.840 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:30.850 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T03:51:30.851 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T03:51:30.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:30.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.987+0000 7f192b99e700 1 Processor -- start 2023-06-08T03:51:30.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.987+0000 7f192b99e700 1 -- start start 2023-06-08T03:51:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192b99e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19240b1d20 0x7f192415c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192b99e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f192406ae80 0x7f19240ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192b99e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19240d2420 0x7f192406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192b99e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240b20f0 con 0x7f19240b1d20 2023-06-08T03:51:30.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192b99e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192406b310 con 0x7f19240d2420 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192b99e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192406b5f0 con 0x7f192406ae80 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192973a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19240b1d20 0x7f192415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192973a700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19240b1d20 0x7f192415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36904/0 (socket says 172.21.15.79:36904) 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.988+0000 7f192973a700 1 -- 172.21.15.79:0/1929263810 learned_addr learned my addr 172.21.15.79:0/1929263810 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f1928f39700 1 --2- 172.21.15.79:0/1929263810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f192406ae80 0x7f19240ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f1929f3b700 1 --2- 172.21.15.79:0/1929263810 >> v2:172.21.15.156:3300/0 conn(0x7f19240d2420 0x7f192406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192973a700 1 -- 172.21.15.79:0/1929263810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f192406ae80 msgr2=0x7f19240ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:30.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192973a700 1 --2- 172.21.15.79:0/1929263810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f192406ae80 0x7f19240ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192973a700 1 -- 172.21.15.79:0/1929263810 >> v2:172.21.15.156:3300/0 conn(0x7f19240d2420 msgr2=0x7f192406c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192973a700 1 --2- 172.21.15.79:0/1929263810 >> v2:172.21.15.156:3300/0 conn(0x7f19240d2420 0x7f192406c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192973a700 1 -- 172.21.15.79:0/1929263810 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192406b8d0 con 0x7f19240b1d20 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192973a700 1 --2- 172.21.15.79:0/1929263810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240b1d20 0x7f192415c710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f19240737f0 tx=0x7f1914009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16817ff6c5eabb23 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f191bfff700 1 -- 172.21.15.79:0/1929263810 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191400b870 con 0x7f19240b1d20 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f191bfff700 1 -- 172.21.15.79:0/1929263810 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191400b9d0 con 0x7f19240b1d20 2023-06-08T03:51:30.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f191bfff700 1 -- 172.21.15.79:0/1929263810 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191400bd60 con 0x7f19240b1d20 2023-06-08T03:51:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 -- 172.21.15.79:0/1929263810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240b1d20 msgr2=0x7f192415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 --2- 172.21.15.79:0/1929263810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240b1d20 0x7f192415c710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f19240737f0 tx=0x7f1914009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 -- 172.21.15.79:0/1929263810 shutdown_connections 2023-06-08T03:51:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 --2- 172.21.15.79:0/1929263810 >> v2:172.21.15.156:3300/0 conn(0x7f19240d2420 0x7f192406c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 --2- 172.21.15.79:0/1929263810 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240b1d20 0x7f192415c710 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 --2- 172.21.15.79:0/1929263810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f192406ae80 0x7f19240ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.989+0000 7f192b99e700 1 -- 172.21.15.79:0/1929263810 >> 172.21.15.79:0/1929263810 conn(0x7f192409bbb0 msgr2=0x7f19240d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.990+0000 7f192b99e700 1 -- 172.21.15.79:0/1929263810 shutdown_connections 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.990+0000 7f192b99e700 1 -- 172.21.15.79:0/1929263810 wait complete. 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.991+0000 7f192b99e700 1 Processor -- start 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.991+0000 7f192b99e700 1 -- start start 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.991+0000 7f192b99e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f192406ae80 0x7f1924154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:30.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.991+0000 7f192b99e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19240b1d20 0x7f19240c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.991+0000 7f192b99e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 0x7f1924154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192b99e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b5210 con 0x7f19240d2420 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192b99e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c6fa0 con 0x7f192406ae80 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192b99e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240c7e00 con 0x7f19240b1d20 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 0x7f1924154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 0x7f1924154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36914/0 (socket says 172.21.15.79:36914) 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 -- 172.21.15.79:0/3146128222 learned_addr learned my addr 172.21.15.79:0/3146128222 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:31.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1928f39700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19240b1d20 0x7f19240c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192973a700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f192406ae80 0x7f1924154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 -- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19240b1d20 msgr2=0x7f19240c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19240b1d20 0x7f19240c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 -- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f192406ae80 msgr2=0x7f1924154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f192406ae80 0x7f1924154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 -- 172.21.15.79:0/3146128222 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914009550 con 0x7f19240d2420 2023-06-08T03:51:31.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192973a700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f192406ae80 0x7f1924154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1929f3b700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 0x7f1924154100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f191c00cc30 tx=0x7f191c00a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00ab70 con 0x7f19240d2420 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00acd0 con 0x7f19240d2420 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1924154640 con 0x7f19240d2420 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c003c40 con 0x7f19240d2420 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.992+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240c74d0 con 0x7f19240d2420 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.993+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1924073e90 con 0x7f19240d2420 2023-06-08T03:51:31.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.994+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f191c00f3f0 con 0x7f19240d2420 2023-06-08T03:51:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.994+0000 7f1919ffb700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f190806ace0 0x7f190806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.994+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f191c089950 con 0x7f19240d2420 2023-06-08T03:51:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.994+0000 7f192973a700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f190806ace0 0x7f190806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.995+0000 7f192973a700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f190806ace0 0x7f190806d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1914000c00 tx=0x7f1914008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:30.998+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f191c054710 con 0x7f19240d2420 2023-06-08T03:51:31.188 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.185+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19240c7f40 con 0x7f19240d2420 2023-06-08T03:51:31.189 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.186+0000 7f1919ffb700 1 -- 172.21.15.79:0/3146128222 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f191c01d090 con 0x7f19240d2420 2023-06-08T03:51:31.190 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:31.190 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:31.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f190806ace0 msgr2=0x7f190806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f190806ace0 0x7f190806d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1914000c00 tx=0x7f1914008030 comp rx=0 tx=0).stop 2023-06-08T03:51:31.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 msgr2=0x7f1924154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.192 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 0x7f1924154100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f191c00cc30 tx=0x7f191c00a1f0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 shutdown_connections 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19240d2420 0x7f1924154100 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f192406ae80 0x7f1924154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f190806ace0 0x7f190806d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 --2- 172.21.15.79:0/3146128222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19240b1d20 0x7f19240c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 >> 172.21.15.79:0/3146128222 conn(0x7f192409bbb0 msgr2=0x7f19240cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 shutdown_connections 2023-06-08T03:51:31.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.188+0000 7f192b99e700 1 -- 172.21.15.79:0/3146128222 wait complete. 2023-06-08T03:51:31.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:31.205 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:31.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.323+0000 7fdbdca67700 1 Processor -- start 2023-06-08T03:51:31.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.323+0000 7fdbdca67700 1 -- start start 2023-06-08T03:51:31.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbdca67700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdbd80d0240 0x7fdbd80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbdca67700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd815d930 con 0x7fdbd80ce3f0 2023-06-08T03:51:31.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbdca67700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd815e770 con 0x7fdbd80cd000 2023-06-08T03:51:31.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbdca67700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd815db60 con 0x7fdbd80d0240 2023-06-08T03:51:31.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd6d9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdbd80ce3f0 0x7fdbd80ce870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60394/0 (socket says 172.21.15.79:60394) 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd6d9d700 1 -- 172.21.15.79:0/3664136629 learned_addr learned my addr 172.21.15.79:0/3664136629 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd659c700 1 --2- 172.21.15.79:0/3664136629 >> v2:172.21.15.156:3300/0 conn(0x7fdbd80d0240 0x7fdbd80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59325719 0 0) 0x7fdbd815d930 con 0x7fdbd80ce3f0 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbc00035b0 con 0x7fdbd80ce3f0 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341395467 0 0) 0x7fdbc00035b0 con 0x7fdbd80ce3f0 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 >> v2:172.21.15.156:3300/0 conn(0x7fdbd80d0240 msgr2=0x7fdbd80af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd559a700 1 --2- 172.21.15.79:0/3664136629 >> v2:172.21.15.156:3300/0 conn(0x7fdbd80d0240 0x7fdbd80af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.324+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 >> v1:172.21.15.156:6789/0 conn(0x7fdbd80cd000 legacy=0x7fdbd80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd815de40 con 0x7fdbd80ce3f0 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fdbd00019f0 con 0x7fdbd80ce3f0 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 >> v1:172.21.15.79:6789/0 conn(0x7fdbd80ce3f0 legacy=0x7fdbd80ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0003a00 0x7fdbc0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbc0004850 0x7fdbc0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbc0005200 0x7fdbc000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.332 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815db60 con 0x7fdbc0004850 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc000c080 con 0x7fdbc0003a00 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc000c240 con 0x7fdbc0005200 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbc0004850 0x7fdbc0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd659c700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0003a00 0x7fdbc0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd6d9d700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbc0005200 0x7fdbc000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 -- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbc0005200 msgr2=0x7fdbc000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbc0005200 0x7fdbc000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.333 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 -- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0003a00 msgr2=0x7fdbc0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0003a00 0x7fdbc0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 -- 172.21.15.79:0/3664136629 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc00035b0 con 0x7fdbc0004850 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd6d9d700 1 -- 172.21.15.79:0/3664136629 reap_dead start 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd659c700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0003a00 0x7fdbc0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd5d9b700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbc0004850 0x7fdbc0004cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80cdb80 tx=0x7fdbc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fda6d44fb259d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80052d0 con 0x7fdbc0004850 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8005450 con 0x7fdbc0004850 2023-06-08T03:51:31.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.325+0000 7fdbd559a700 1 -- 172.21.15.79:0/3664136629 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80038c0 con 0x7fdbc0004850 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 -- 172.21.15.79:0/3664136629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbc0004850 msgr2=0x7fdbc0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbc0004850 0x7fdbc0004cc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80cdb80 tx=0x7fdbc0005880 comp rx=0 tx=0).stop 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 -- 172.21.15.79:0/3664136629 shutdown_connections 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3664136629 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbc0004850 0x7fdbc0004cc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 -- 172.21.15.79:0/3664136629 >> 172.21.15.79:0/3664136629 conn(0x7fdbd806e1e0 msgr2=0x7fdbd806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 -- 172.21.15.79:0/3664136629 shutdown_connections 2023-06-08T03:51:31.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.326+0000 7fdbdca67700 1 -- 172.21.15.79:0/3664136629 wait complete. 2023-06-08T03:51:31.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.327+0000 7fdbdca67700 1 Processor -- start 2023-06-08T03:51:31.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.327+0000 7fdbdca67700 1 -- start start 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbdca67700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0004850 0x7fdbd81f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbdca67700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd815b0e0 0x7fdbd815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbdca67700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 0x7fdbd81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbdca67700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815ba70 con 0x7fdbd8158480 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbdca67700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8158e30 con 0x7fdbc0004850 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbdca67700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815bc80 con 0x7fdbd815b0e0 2023-06-08T03:51:31.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 0x7fdbd81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 0x7fdbd81588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36932/0 (socket says 172.21.15.79:36932) 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 -- 172.21.15.79:0/3162446269 learned_addr learned my addr 172.21.15.79:0/3162446269 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd5d9b700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd815b0e0 0x7fdbd815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd659c700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0004850 0x7fdbd81f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 -- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd815b0e0 msgr2=0x7fdbd815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd815b0e0 0x7fdbd815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 -- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0004850 msgr2=0x7fdbd81f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0004850 0x7fdbd81f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 -- 172.21.15.79:0/3162446269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd815bf60 con 0x7fdbd8158480 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd5d9b700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd815b0e0 0x7fdbd815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.328+0000 7fdbd6d9d700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 0x7fdbd81588f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8072150 tx=0x7fdbd0007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.329+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0017070 con 0x7fdbd8158480 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.329+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd000ec60 con 0x7fdbd8158480 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.329+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd815c240 con 0x7fdbd8158480 2023-06-08T03:51:31.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.329+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000ee10 con 0x7fdbd8158480 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.329+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd815c800 con 0x7fdbd8158480 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.330+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd8073e90 con 0x7fdbd8158480 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.330+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbd0013420 con 0x7fdbd8158480 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.330+0000 7fdbc6ffd700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdbb806ace0 0x7fdbb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.331+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdbd00894a0 con 0x7fdbd8158480 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.331+0000 7fdbd659c700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdbb806ace0 0x7fdbb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.331+0000 7fdbd659c700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdbb806ace0 0x7fdbb806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdbd804fa30 tx=0x7fdbcc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.342 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.335+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdbd0054260 con 0x7fdbd8158480 2023-06-08T03:51:31.524 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.520+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdbd80b3c90 con 0x7fdbd8158480 2023-06-08T03:51:31.525 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.521+0000 7fdbc6ffd700 1 -- 172.21.15.79:0/3162446269 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdbd0057880 con 0x7fdbd8158480 2023-06-08T03:51:31.525 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:31.526 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdbb806ace0 msgr2=0x7fdbb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdbb806ace0 0x7fdbb806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdbd804fa30 tx=0x7fdbcc008040 comp rx=0 tx=0).stop 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 msgr2=0x7fdbd81588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 0x7fdbd81588f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8072150 tx=0x7fdbd0007c90 comp rx=0 tx=0).stop 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 shutdown_connections 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdbd8158480 0x7fdbd81588f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbc0004850 0x7fdbd81f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.527 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdbb806ace0 0x7fdbb806d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 --2- 172.21.15.79:0/3162446269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdbd815b0e0 0x7fdbd815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.522+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 >> 172.21.15.79:0/3162446269 conn(0x7fdbd806e1e0 msgr2=0x7fdbd80b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:31.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.523+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 shutdown_connections 2023-06-08T03:51:31.528 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.523+0000 7fdbdca67700 1 -- 172.21.15.79:0/3162446269 wait complete. 2023-06-08T03:51:31.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:31.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.656+0000 7f600c161700 1 Processor -- start 2023-06-08T03:51:31.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.656+0000 7f600c161700 1 -- start start 2023-06-08T03:51:31.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.657+0000 7f600c161700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f600415a3a0 0x7f600415a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.657+0000 7f600c161700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 0x7f600406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.657+0000 7f600c161700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60040b47e0 con 0x7f60040b4410 2023-06-08T03:51:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.657+0000 7f600c161700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600415ad80 con 0x7f600415a3a0 2023-06-08T03:51:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.657+0000 7f600c161700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600415af80 con 0x7f600406e710 2023-06-08T03:51:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6009efd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f60040b4410 0x7f60040c9ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60412/0 (socket says 172.21.15.79:60412) 2023-06-08T03:51:31.662 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6009efd700 1 -- 172.21.15.79:0/3946434870 learned_addr learned my addr 172.21.15.79:0/3946434870 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f60096fc700 1 --2- 172.21.15.79:0/3946434870 >> v2:172.21.15.156:3300/0 conn(0x7f600415a3a0 0x7f600415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f600a6fe700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 0x7f600406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124863084 0 0) 0x7f60040b47e0 con 0x7f60040b4410 2023-06-08T03:51:31.663 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff00035b0 con 0x7f60040b4410 2023-06-08T03:51:31.666 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100198491 0 0) 0x7f5ff00035b0 con 0x7f60040b4410 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 msgr2=0x7f600406eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 0x7f600406eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 >> v2:172.21.15.156:3300/0 conn(0x7f600415a3a0 msgr2=0x7f600415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 --2- 172.21.15.79:0/3946434870 >> v2:172.21.15.156:3300/0 conn(0x7f600415a3a0 0x7f600415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600415b260 con 0x7f60040b4410 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f5ff4001670 con 0x7f60040b4410 2023-06-08T03:51:31.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 >> v1:172.21.15.79:6789/0 conn(0x7f60040b4410 legacy=0x7f60040c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 0x7f5ff0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0003f80 0x7f5ff0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff0004d90 0x7f5ff0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600415ad80 con 0x7f5ff0004d90 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600415af80 con 0x7f5ff0003f80 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff000ba10 con 0x7f600406e710 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f600a6fe700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff0004d90 0x7f5ff0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f6009efd700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 0x7f5ff0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.658+0000 7f60096fc700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0003f80 0x7f5ff0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 -- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 msgr2=0x7f5ff0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f600406e710 0x7f5ff0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 -- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0003f80 msgr2=0x7f5ff0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0003f80 0x7f5ff0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 -- 172.21.15.79:0/3946434870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00035b0 con 0x7f5ff0004d90 2023-06-08T03:51:31.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff0004d90 0x7f5ff0005200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009fc0 tx=0x7f5ff0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a714bdd7eb4ae3 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00ec30 con 0x7f5ff0004d90 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc00f040 con 0x7f5ff0004d90 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f6008efb700 1 -- 172.21.15.79:0/3946434870 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc014480 con 0x7f5ff0004d90 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600c161700 1 -- 172.21.15.79:0/3946434870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff0004d90 msgr2=0x7f5ff0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600c161700 1 --2- 172.21.15.79:0/3946434870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ff0004d90 0x7f5ff0005200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009fc0 tx=0x7f5ff0005880 comp rx=0 tx=0).stop 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600a6fe700 1 -- 172.21.15.79:0/3946434870 reap_dead start 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.659+0000 7f600c161700 1 -- 172.21.15.79:0/3946434870 shutdown_connections 2023-06-08T03:51:31.670 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.660+0000 7f600c161700 1 -- 172.21.15.79:0/3946434870 >> 172.21.15.79:0/3946434870 conn(0x7f60040cf940 msgr2=0x7f60040cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.660+0000 7f600c161700 1 -- 172.21.15.79:0/3946434870 shutdown_connections 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.660+0000 7f600c161700 1 -- 172.21.15.79:0/3946434870 wait complete. 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.661+0000 7f600c161700 1 Processor -- start 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.661+0000 7f600c161700 1 -- start start 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.661+0000 7f600c161700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0004d90 0x7f600414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.661+0000 7f600c161700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60041569b0 0x7f6004156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.661+0000 7f600c161700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 0x7f600415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600c161700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60041560f0 con 0x7f600415bb40 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600c161700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004157360 con 0x7f5ff0004d90 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600c161700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60041577e0 con 0x7f60041569b0 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 0x7f600415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 0x7f600415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36952/0 (socket says 172.21.15.79:36952) 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 -- 172.21.15.79:0/2483361956 learned_addr learned my addr 172.21.15.79:0/2483361956 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f6009efd700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0004d90 0x7f600414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f60096fc700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60041569b0 0x7f6004156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 -- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60041569b0 msgr2=0x7f6004156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60041569b0 0x7f6004156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 -- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0004d90 msgr2=0x7f600414a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0004d90 0x7f600414a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 -- 172.21.15.79:0/2483361956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004155850 con 0x7f600415bb40 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f6009efd700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0004d90 0x7f600414a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600a6fe700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 0x7f600415bfe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc016000 tx=0x7f5ffc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc007000 con 0x7f600415bb40 2023-06-08T03:51:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc003e50 con 0x7f600415bb40 2023-06-08T03:51:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6004155b30 con 0x7f600415bb40 2023-06-08T03:51:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc014b50 con 0x7f600415bb40 2023-06-08T03:51:31.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.662+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600415b620 con 0x7f600415bb40 2023-06-08T03:51:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.663+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004073e90 con 0x7f600415bb40 2023-06-08T03:51:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.664+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ffc023020 con 0x7f600415bb40 2023-06-08T03:51:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.664+0000 7f5ffa7fc700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5fe806ad10 0x7f5fe806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.664+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ffc08ab60 con 0x7f600415bb40 2023-06-08T03:51:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.664+0000 7f6009efd700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5fe806ad10 0x7f5fe806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.665+0000 7f6009efd700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5fe806ad10 0x7f5fe806d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4000fb0 tx=0x7f5ff4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.676 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.669+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5ffc0559b0 con 0x7f600415bb40 2023-06-08T03:51:31.858 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.854+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6004072b60 con 0x7f600415bb40 2023-06-08T03:51:31.859 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.855+0000 7f5ffa7fc700 1 -- 172.21.15.79:0/2483361956 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ffc058fd0 con 0x7f600415bb40 2023-06-08T03:51:31.859 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:31.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5fe806ad10 msgr2=0x7f5fe806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5fe806ad10 0x7f5fe806d1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4000fb0 tx=0x7f5ff4009040 comp rx=0 tx=0).stop 2023-06-08T03:51:31.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 msgr2=0x7f600415bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:31.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 0x7f600415bfe0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc016000 tx=0x7f5ffc009750 comp rx=0 tx=0).stop 2023-06-08T03:51:31.861 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 shutdown_connections 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f600415bb40 0x7f600415bfe0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ff0004d90 0x7f600414a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5fe806ad10 0x7f5fe806d1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 --2- 172.21.15.79:0/2483361956 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f60041569b0 0x7f6004156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 >> 172.21.15.79:0/2483361956 conn(0x7f60040cf940 msgr2=0x7f600406ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 shutdown_connections 2023-06-08T03:51:31.862 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.857+0000 7f600c161700 1 -- 172.21.15.79:0/2483361956 wait complete. 2023-06-08T03:51:31.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:31.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.991+0000 7f2c11c9e700 1 Processor -- start 2023-06-08T03:51:31.995 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.991+0000 7f2c11c9e700 1 -- start start 2023-06-08T03:51:31.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c11c9e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2c0c158ec0 0x7f2c0c15d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.997 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c11c9e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c11c9e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c15d8d0 con 0x7f2c0c071070 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c11c9e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c0c06df90 con 0x7f2c0c158ec0 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c11c9e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0ca320 con 0x7f2c0c0cc650 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c0b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2c0c158ec0 0x7f2c0c15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c0b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2c0c158ec0 0x7f2c0c15d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36956/0 (socket says 172.21.15.79:36956) 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.992+0000 7f2c0b7fe700 1 -- 172.21.15.79:0/1468031795 learned_addr learned my addr 172.21.15.79:0/1468031795 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:31.998 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0bfff700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0b7fe700 1 -- 172.21.15.79:0/1468031795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 msgr2=0x7f2c0c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0b7fe700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0b7fe700 1 -- 172.21.15.79:0/1468031795 >> v1:172.21.15.156:6789/0 conn(0x7f2c0c071070 legacy=0x7f2c0c0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0b7fe700 1 -- 172.21.15.79:0/1468031795 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c159290 con 0x7f2c0c158ec0 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0bfff700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0b7fe700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c15d390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2c000098f0 tx=0x7f2c00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d4158da0570310 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0a7fc700 1 -- 172.21.15.79:0/1468031795 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00004230 con 0x7f2c0c158ec0 2023-06-08T03:51:31.999 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0a7fc700 1 -- 172.21.15.79:0/1468031795 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00004390 con 0x7f2c0c158ec0 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c0a7fc700 1 -- 172.21.15.79:0/1468031795 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000036a0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1468031795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 msgr2=0x7f2c0c15d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c15d390 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2c000098f0 tx=0x7f2c00009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1468031795 shutdown_connections 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1468031795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c15d390 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.000 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.993+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1468031795 >> 172.21.15.79:0/1468031795 conn(0x7f2c0c0b1d30 msgr2=0x7f2c0c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:32.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.994+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1468031795 shutdown_connections 2023-06-08T03:51:32.001 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.994+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1468031795 wait complete. 2023-06-08T03:51:32.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.995+0000 7f2c11c9e700 1 Processor -- start 2023-06-08T03:51:32.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.995+0000 7f2c11c9e700 1 -- start start 2023-06-08T03:51:32.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.995+0000 7f2c11c9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c0c071070 0x7f2c0c0c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.003 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0c55a0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0c8d90 con 0x7f2c0c071070 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0c4920 con 0x7f2c0c0cc650 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c14ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36966/0 (socket says 172.21.15.79:36966) 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 -- 172.21.15.79:0/1879163006 learned_addr learned my addr 172.21.15.79:0/1879163006 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0b7fe700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c0c071070 0x7f2c0c0c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0affd700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 -- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 msgr2=0x7f2c0c0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 -- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c0c071070 msgr2=0x7f2c0c0c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c0c071070 0x7f2c0c0c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 -- 172.21.15.79:0/1879163006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c00009550 con 0x7f2c0c158ec0 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0b7fe700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c0c071070 0x7f2c0c0c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c0bfff700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c14ad50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400ceb0 tx=0x7f2c0400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0400ae10 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c04015440 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c040157f0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c0c4c00 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.996+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c0c51c0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.997+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c0c56e0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.998+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c0400d430 con 0x7f2c0c158ec0 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.998+0000 7f2bf3fff700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2bf806ace0 0x7f2bf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.998+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2c04089d50 con 0x7f2c0c158ec0 2023-06-08T03:51:32.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.998+0000 7f2c0b7fe700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2bf806ace0 0x7f2bf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:31.999+0000 7f2c0b7fe700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2bf806ace0 0x7f2bf806d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009130 tx=0x7f2c00008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.003+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2c0c0c56e0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:51:32.015 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:51:32.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.193+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c0c0c56e0 con 0x7f2c0c158ec0 2023-06-08T03:51:32.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.193+0000 7f2bf3fff700 1 -- 172.21.15.79:0/1879163006 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c0401a020 con 0x7f2c0c158ec0 2023-06-08T03:51:32.198 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:32.199 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.195+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2bf806ace0 msgr2=0x7f2bf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.195+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2bf806ace0 0x7f2bf806d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009130 tx=0x7f2c00008030 comp rx=0 tx=0).stop 2023-06-08T03:51:32.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.195+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 msgr2=0x7f2c0c14ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.195+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c14ad50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400ceb0 tx=0x7f2c0400cf90 comp rx=0 tx=0).stop 2023-06-08T03:51:32.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 shutdown_connections 2023-06-08T03:51:32.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2c0c158ec0 0x7f2c0c14ad50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c0c071070 0x7f2c0c0c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2bf806ace0 0x7f2bf806d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 --2- 172.21.15.79:0/1879163006 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c0c0cc650 0x7f2c0c0c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 >> 172.21.15.79:0/1879163006 conn(0x7f2c0c0b1d30 msgr2=0x7f2c0c15c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:32.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 shutdown_connections 2023-06-08T03:51:32.201 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.196+0000 7f2c11c9e700 1 -- 172.21.15.79:0/1879163006 wait complete. 2023-06-08T03:51:32.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:32.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.330+0000 7f5a0938c700 1 Processor -- start 2023-06-08T03:51:32.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.330+0000 7f5a0938c700 1 -- start start 2023-06-08T03:51:32.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a0938c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5a0415c8e0 0x7f5a0406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a0938c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a04161010 0x7f5a040b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a0938c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0415ccb0 con 0x7f5a040ce3c0 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a0938c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0406cc50 con 0x7f5a0415c8e0 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a0938c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a04160530 con 0x7f5a04161010 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5a0415c8e0 0x7f5a0406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5a0415c8e0 0x7f5a0406c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36970/0 (socket says 172.21.15.79:36970) 2023-06-08T03:51:32.336 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 -- 172.21.15.79:0/864325203 learned_addr learned my addr 172.21.15.79:0/864325203 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 -- 172.21.15.79:0/864325203 >> v2:172.21.15.156:3300/0 conn(0x7f5a04161010 msgr2=0x7f5a040b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 --2- 172.21.15.79:0/864325203 >> v2:172.21.15.156:3300/0 conn(0x7f5a04161010 0x7f5a040b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 -- 172.21.15.79:0/864325203 >> v1:172.21.15.156:6789/0 conn(0x7f5a040ce3c0 legacy=0x7f5a0406b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 -- 172.21.15.79:0/864325203 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04160810 con 0x7f5a0415c8e0 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a02ffd700 1 --2- 172.21.15.79:0/864325203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0415c8e0 0x7f5a0406c590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a040666e0 tx=0x7f59f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aba4404eaa78bb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a01ffb700 1 -- 172.21.15.79:0/864325203 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8004230 con 0x7f5a0415c8e0 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a01ffb700 1 -- 172.21.15.79:0/864325203 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8004390 con 0x7f5a0415c8e0 2023-06-08T03:51:32.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.331+0000 7f5a01ffb700 1 -- 172.21.15.79:0/864325203 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80036a0 con 0x7f5a0415c8e0 2023-06-08T03:51:32.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 -- 172.21.15.79:0/864325203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0415c8e0 msgr2=0x7f5a0406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 --2- 172.21.15.79:0/864325203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0415c8e0 0x7f5a0406c590 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a040666e0 tx=0x7f59f8009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 -- 172.21.15.79:0/864325203 shutdown_connections 2023-06-08T03:51:32.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 --2- 172.21.15.79:0/864325203 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a0415c8e0 0x7f5a0406c590 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 --2- 172.21.15.79:0/864325203 >> v2:172.21.15.156:3300/0 conn(0x7f5a04161010 0x7f5a040b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 -- 172.21.15.79:0/864325203 >> 172.21.15.79:0/864325203 conn(0x7f5a0406e440 msgr2=0x7f5a0406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:32.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 -- 172.21.15.79:0/864325203 shutdown_connections 2023-06-08T03:51:32.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.332+0000 7f5a0938c700 1 -- 172.21.15.79:0/864325203 wait complete. 2023-06-08T03:51:32.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.333+0000 7f5a0938c700 1 Processor -- start 2023-06-08T03:51:32.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 -- start start 2023-06-08T03:51:32.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 0x7f5a0415e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0415c8e0 0x7f5a0415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a04161010 0x7f5a041541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0415d590 con 0x7f5a040ce3c0 2023-06-08T03:51:32.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0415d870 con 0x7f5a04161010 2023-06-08T03:51:32.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a0938c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0415db50 con 0x7f5a0415c8e0 2023-06-08T03:51:32.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a02ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 0x7f5a0415e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a02ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 0x7f5a0415e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36976/0 (socket says 172.21.15.79:36976) 2023-06-08T03:51:32.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.334+0000 7f5a02ffd700 1 -- 172.21.15.79:0/1489611925 learned_addr learned my addr 172.21.15.79:0/1489611925 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:32.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a02ffd700 1 -- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0415c8e0 msgr2=0x7f5a0415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a02ffd700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0415c8e0 0x7f5a0415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a037fe700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a04161010 0x7f5a041541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a02ffd700 1 -- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a04161010 msgr2=0x7f5a041541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.348 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a02ffd700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a04161010 0x7f5a041541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a02ffd700 1 -- 172.21.15.79:0/1489611925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009550 con 0x7f5a040ce3c0 2023-06-08T03:51:32.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a02ffd700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 0x7f5a0415e940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5a040b3470 tx=0x7f59f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8003dc0 con 0x7f5a040ce3c0 2023-06-08T03:51:32.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f801f070 con 0x7f5a040ce3c0 2023-06-08T03:51:32.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a0938c700 1 -- 172.21.15.79:0/1489611925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a04158960 con 0x7f5a040ce3c0 2023-06-08T03:51:32.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8011870 con 0x7f5a040ce3c0 2023-06-08T03:51:32.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.335+0000 7f5a0938c700 1 -- 172.21.15.79:0/1489611925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a04158580 con 0x7f5a040ce3c0 2023-06-08T03:51:32.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.336+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc0052f0 con 0x7f5a040ce3c0 2023-06-08T03:51:32.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.336+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f59f8021030 con 0x7f5a040ce3c0 2023-06-08T03:51:32.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.337+0000 7f59eb7fe700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f59f006ace0 0x7f59f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.337+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f59f808ae30 con 0x7f5a040ce3c0 2023-06-08T03:51:32.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.337+0000 7f5a027fc700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f59f006ace0 0x7f59f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.338+0000 7f5a027fc700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f59f006ace0 0x7f59f006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5a040cf890 tx=0x7f59f4006c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.341+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f59f8055c80 con 0x7f5a040ce3c0 2023-06-08T03:51:32.532 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.528+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59dc005b20 con 0x7f5a040ce3c0 2023-06-08T03:51:32.533 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.529+0000 7f59eb7fe700 1 -- 172.21.15.79:0/1489611925 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59f80592a0 con 0x7f5a040ce3c0 2023-06-08T03:51:32.533 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f59f006ace0 msgr2=0x7f59f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f59f006ace0 0x7f59f006d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5a040cf890 tx=0x7f59f4006c80 comp rx=0 tx=0).stop 2023-06-08T03:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 msgr2=0x7f5a0415e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 0x7f5a0415e940 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5a040b3470 tx=0x7f59f8004750 comp rx=0 tx=0).stop 2023-06-08T03:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 shutdown_connections 2023-06-08T03:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a04161010 0x7f5a041541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5a040ce3c0 0x7f5a0415e940 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f59f006ace0 0x7f59f006d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 --2- 172.21.15.79:0/1489611925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a0415c8e0 0x7f5a0415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 >> 172.21.15.79:0/1489611925 conn(0x7f5a0406e440 msgr2=0x7f5a04156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 shutdown_connections 2023-06-08T03:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.531+0000 7f59e97fa700 1 -- 172.21.15.79:0/1489611925 wait complete. 2023-06-08T03:51:32.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:32.668 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.664+0000 7f06685f6700 1 Processor -- start 2023-06-08T03:51:32.669 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.664+0000 7f06685f6700 1 -- start start 2023-06-08T03:51:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.665+0000 7f06685f6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0660158d50 0x7f06600ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.665+0000 7f06685f6700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f066015d3d0 0x7f06600cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.665+0000 7f06685f6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06600cc920 con 0x7f06600cc4c0 2023-06-08T03:51:32.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.665+0000 7f06685f6700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600cd7b0 con 0x7f066015d3d0 2023-06-08T03:51:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.665+0000 7f06685f6700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600cda50 con 0x7f0660158d50 2023-06-08T03:51:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0666392700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0660158d50 0x7f06600ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f066015d3d0 0x7f06600cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f066015d3d0 0x7f06600cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:59882/0 (socket says 172.21.15.79:59882) 2023-06-08T03:51:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0666b93700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f06600cc4c0 0x7f066006eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:34392/0 (socket says 172.21.15.79:34392) 2023-06-08T03:51:32.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 -- 172.21.15.79:0/4098802686 learned_addr learned my addr 172.21.15.79:0/4098802686 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 -- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0660158d50 msgr2=0x7f06600ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 --2- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0660158d50 0x7f06600ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 -- 172.21.15.79:0/4098802686 >> v1:172.21.15.156:6789/0 conn(0x7f06600cc4c0 legacy=0x7f066006eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 -- 172.21.15.79:0/4098802686 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600cdd30 con 0x7f066015d3d0 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665b91700 1 --2- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f066015d3d0 0x7f06600cbf80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f065000bb90 tx=0x7f065000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8da20356d8c70f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665390700 1 -- 172.21.15.79:0/4098802686 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065000dd30 con 0x7f066015d3d0 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665390700 1 -- 172.21.15.79:0/4098802686 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065000de90 con 0x7f066015d3d0 2023-06-08T03:51:32.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.666+0000 7f0665390700 1 -- 172.21.15.79:0/4098802686 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650010690 con 0x7f066015d3d0 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 -- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f066015d3d0 msgr2=0x7f06600cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 --2- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f066015d3d0 0x7f06600cbf80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f065000bb90 tx=0x7f065000bea0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 -- 172.21.15.79:0/4098802686 shutdown_connections 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 --2- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0660158d50 0x7f06600ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 --2- 172.21.15.79:0/4098802686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f066015d3d0 0x7f06600cbf80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 -- 172.21.15.79:0/4098802686 >> 172.21.15.79:0/4098802686 conn(0x7f06600b1d30 msgr2=0x7f066006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:32.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 -- 172.21.15.79:0/4098802686 shutdown_connections 2023-06-08T03:51:32.675 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.667+0000 7f06685f6700 1 -- 172.21.15.79:0/4098802686 wait complete. 2023-06-08T03:51:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.668+0000 7f06685f6700 1 Processor -- start 2023-06-08T03:51:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.668+0000 7f06685f6700 1 -- start start 2023-06-08T03:51:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f06685f6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600cc4c0 0x7f06600c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f06685f6700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 0x7f0660148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f06685f6700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f066015d3d0 0x7f06600b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.677 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f06685f6700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600cb8a0 con 0x7f0660158d50 2023-06-08T03:51:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f06685f6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600cbb80 con 0x7f06600cc4c0 2023-06-08T03:51:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f06685f6700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600cbe60 con 0x7f066015d3d0 2023-06-08T03:51:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 0x7f0660148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 0x7f0660148910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36992/0 (socket says 172.21.15.79:36992) 2023-06-08T03:51:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 -- 172.21.15.79:0/1889056224 learned_addr learned my addr 172.21.15.79:0/1889056224 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:32.678 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0666392700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600cc4c0 0x7f06600c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0666b93700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f066015d3d0 0x7f06600b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 -- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f066015d3d0 msgr2=0x7f06600b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f066015d3d0 0x7f06600b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 -- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600cc4c0 msgr2=0x7f06600c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600cc4c0 0x7f06600c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0665b91700 1 -- 172.21.15.79:0/1889056224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065000b7f0 con 0x7f0660158d50 2023-06-08T03:51:32.679 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.669+0000 7f0666b93700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f066015d3d0 0x7f06600b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.670+0000 7f0665b91700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 0x7f0660148910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0650000c00 tx=0x7f0650004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.670+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065001f070 con 0x7f0660158d50 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.670+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650003970 con 0x7f0660158d50 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.670+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06600cc140 con 0x7f0660158d50 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.670+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650003d20 con 0x7f0660158d50 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.670+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06600cb550 con 0x7f0660158d50 2023-06-08T03:51:32.680 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.671+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0660073e90 con 0x7f0660158d50 2023-06-08T03:51:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.671+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0650019400 con 0x7f0660158d50 2023-06-08T03:51:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.671+0000 7f0656ffd700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f064406ace0 0x7f064406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.672+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f065008ca60 con 0x7f0660158d50 2023-06-08T03:51:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.672+0000 7f0666392700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f064406ace0 0x7f064406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.672+0000 7f0666392700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f064406ace0 0x7f064406d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f065c009be0 tx=0x7f065c009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:32.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.676+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f06500578b0 con 0x7f0660158d50 2023-06-08T03:51:32.865 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.861+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f066006d9a0 con 0x7f0660158d50 2023-06-08T03:51:32.866 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.862+0000 7f0656ffd700 1 -- 172.21.15.79:0/1889056224 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f065005aed0 con 0x7f0660158d50 2023-06-08T03:51:32.866 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f064406ace0 msgr2=0x7f064406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f064406ace0 0x7f064406d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f065c009be0 tx=0x7f065c009210 comp rx=0 tx=0).stop 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 msgr2=0x7f0660148910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 0x7f0660148910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0650000c00 tx=0x7f0650004750 comp rx=0 tx=0).stop 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 shutdown_connections 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f064406ace0 0x7f064406d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.868 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f066015d3d0 0x7f06600b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600cc4c0 0x7f06600c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 --2- 172.21.15.79:0/1889056224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0660158d50 0x7f0660148910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:32.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.863+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 >> 172.21.15.79:0/1889056224 conn(0x7f06600b1d30 msgr2=0x7f06600afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:32.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.864+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 shutdown_connections 2023-06-08T03:51:32.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.864+0000 7f06685f6700 1 -- 172.21.15.79:0/1889056224 wait complete. 2023-06-08T03:51:32.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-08T03:51:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.998+0000 7f28ee660700 1 Processor -- start 2023-06-08T03:51:33.002 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.998+0000 7f28ee660700 1 -- start start 2023-06-08T03:51:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28ee660700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28e80d0740 0x7f28e80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28ee660700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80b4f40 con 0x7f28e80d0b10 2023-06-08T03:51:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28ee660700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80d2e20 con 0x7f28e80b5a50 2023-06-08T03:51:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28ee660700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e80d3100 con 0x7f28e80d0740 2023-06-08T03:51:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f28e80d0740 0x7f28e80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.004 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e77fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f28e80d0b10 0x7f28e80d0f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60476/0 (socket says 172.21.15.79:60476) 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e77fe700 1 -- 172.21.15.79:0/3505331907 learned_addr learned my addr 172.21.15.79:0/3505331907 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e7fff700 1 -- 172.21.15.79:0/3505331907 >> v1:172.21.15.156:6789/0 conn(0x7f28e80b5a50 legacy=0x7f28e80b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e7fff700 1 -- 172.21.15.79:0/3505331907 >> v1:172.21.15.79:6789/0 conn(0x7f28e80d0b10 legacy=0x7f28e80d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e6ffd700 1 -- 172.21.15.79:0/3505331907 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679041564 0 0) 0x7f28e80b4f40 con 0x7f28e80d0b10 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e7fff700 1 -- 172.21.15.79:0/3505331907 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80d33e0 con 0x7f28e80d0740 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e7fff700 1 --2- 172.21.15.79:0/3505331907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80d0740 0x7f28e80cd1c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009840 tx=0x7f28d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa078bac82aa60f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.005 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e6ffd700 1 -- 172.21.15.79:0/3505331907 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8004230 con 0x7f28e80d0740 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e6ffd700 1 -- 172.21.15.79:0/3505331907 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d8004390 con 0x7f28e80d0740 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:32.999+0000 7f28e6ffd700 1 -- 172.21.15.79:0/3505331907 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80036a0 con 0x7f28e80d0740 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 -- 172.21.15.79:0/3505331907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80d0740 msgr2=0x7f28e80cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 --2- 172.21.15.79:0/3505331907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80d0740 0x7f28e80cd1c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009840 tx=0x7f28d8009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 -- 172.21.15.79:0/3505331907 shutdown_connections 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 --2- 172.21.15.79:0/3505331907 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80d0740 0x7f28e80cd1c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 -- 172.21.15.79:0/3505331907 >> 172.21.15.79:0/3505331907 conn(0x7f28e8071480 msgr2=0x7f28e8071850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:33.006 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 -- 172.21.15.79:0/3505331907 shutdown_connections 2023-06-08T03:51:33.007 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.000+0000 7f28ee660700 1 -- 172.21.15.79:0/3505331907 wait complete. 2023-06-08T03:51:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.001+0000 7f28ee660700 1 Processor -- start 2023-06-08T03:51:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.001+0000 7f28ee660700 1 -- start start 2023-06-08T03:51:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28ee660700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 0x7f28e81531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28ee660700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28e80d0740 0x7f28e815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28ee660700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e80d0b10 0x7f28e8159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.009 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28ee660700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815e540 con 0x7f28e80b5a50 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28ee660700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815e820 con 0x7f28e80d0b10 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28ee660700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815eb00 con 0x7f28e80d0740 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28e7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 0x7f28e81531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28e7fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 0x7f28e81531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37004/0 (socket says 172.21.15.79:37004) 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.002+0000 7f28e7fff700 1 -- 172.21.15.79:0/3077799405 learned_addr learned my addr 172.21.15.79:0/3077799405 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28ecbfd700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e80d0b10 0x7f28e8159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.010 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e7fff700 1 -- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28e80d0740 msgr2=0x7f28e815dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e77fe700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28e80d0740 0x7f28e815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e7fff700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28e80d0740 0x7f28e815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e7fff700 1 -- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e80d0b10 msgr2=0x7f28e8159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e7fff700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e80d0b10 0x7f28e8159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e7fff700 1 -- 172.21.15.79:0/3077799405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80094a0 con 0x7f28e80b5a50 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28ecbfd700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e80d0b10 0x7f28e8159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e7fff700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 0x7f28e81531a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f28d80090b0 tx=0x7f28d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.011 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8011830 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d801e070 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e815ede0 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8011bb0 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.003+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e815f5e0 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.004+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e8073e90 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.004+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28d8011d10 con 0x7f28e80b5a50 2023-06-08T03:51:33.012 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.005+0000 7f28e4ff9700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f28d406ace0 0x7f28d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.005+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f28d808ab20 con 0x7f28e80b5a50 2023-06-08T03:51:33.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.005+0000 7f28e77fe700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f28d406ace0 0x7f28d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.005+0000 7f28e77fe700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f28d406ace0 0x7f28d406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b8500 tx=0x7f28dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.013 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.009+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f28d8055970 con 0x7f28e80b5a50 2023-06-08T03:51:33.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.210+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28e8066790 con 0x7f28e80b5a50 2023-06-08T03:51:33.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.210+0000 7f28e4ff9700 1 -- 172.21.15.79:0/3077799405 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28d8020080 con 0x7f28e80b5a50 2023-06-08T03:51:33.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f28d406ace0 msgr2=0x7f28d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f28d406ace0 0x7f28d406d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b8500 tx=0x7f28dc009040 comp rx=0 tx=0).stop 2023-06-08T03:51:33.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 msgr2=0x7f28e81531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 0x7f28e81531a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f28d80090b0 tx=0x7f28d8004750 comp rx=0 tx=0).stop 2023-06-08T03:51:33.217 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 shutdown_connections 2023-06-08T03:51:33.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28e80d0b10 0x7f28e8159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f28e80b5a50 0x7f28e81531a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f28d406ace0 0x7f28d406d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 --2- 172.21.15.79:0/3077799405 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28e80d0740 0x7f28e815dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 >> 172.21.15.79:0/3077799405 conn(0x7f28e8071480 msgr2=0x7f28e81590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:33.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 shutdown_connections 2023-06-08T03:51:33.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.213+0000 7f28ee660700 1 -- 172.21.15.79:0/3077799405 wait complete. 2023-06-08T03:51:33.219 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2023-06-08T03:51:33.228 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi156.front.sepia.ceph.com... 2023-06-08T03:51:33.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:33.229 DEBUG:teuthology.orchestra.run.smithi156:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-06-08T03:51:33.276 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:33.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T03:51:33.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:33.277 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-08T03:51:33.296 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-06-08T03:51:33.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:51:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.412+0000 7f2aaec48700 1 Processor -- start 2023-06-08T03:51:33.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.412+0000 7f2aaec48700 1 -- start start 2023-06-08T03:51:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aaec48700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2aa815a640 0x7f2aa80b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aaec48700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2aa815aa10 0x7f2aa80ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aaec48700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa80b0750 con 0x7f2aa80ce930 2023-06-08T03:51:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aaec48700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa80ceeb0 con 0x7f2aa815aa10 2023-06-08T03:51:33.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aaec48700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa80b09a0 con 0x7f2aa815a640 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aa7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2aa815aa10 0x7f2aa80ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aa7fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2aa815aa10 0x7f2aa80ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37012/0 (socket says 172.21.15.79:37012) 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.413+0000 7f2aa7fff700 1 -- 172.21.15.79:0/2187182861 learned_addr learned my addr 172.21.15.79:0/2187182861 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aac9e4700 1 --2- 172.21.15.79:0/2187182861 >> v2:172.21.15.156:3300/0 conn(0x7f2aa815a640 0x7f2aa80b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa7fff700 1 -- 172.21.15.79:0/2187182861 >> v2:172.21.15.156:3300/0 conn(0x7f2aa815a640 msgr2=0x7f2aa80b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa7fff700 1 --2- 172.21.15.79:0/2187182861 >> v2:172.21.15.156:3300/0 conn(0x7f2aa815a640 0x7f2aa80b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.419 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa7fff700 1 -- 172.21.15.79:0/2187182861 >> v1:172.21.15.156:6789/0 conn(0x7f2aa80ce930 legacy=0x7f2aa80af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa7fff700 1 -- 172.21.15.79:0/2187182861 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80b0c80 con 0x7f2aa815aa10 2023-06-08T03:51:33.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aac9e4700 1 --2- 172.21.15.79:0/2187182861 >> v2:172.21.15.156:3300/0 conn(0x7f2aa815a640 0x7f2aa80b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:33.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa7fff700 1 --2- 172.21.15.79:0/2187182861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa815aa10 0x7f2aa80ce3f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80b33b0 tx=0x7f2a94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd8405dfd73698c server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa77fe700 1 -- 172.21.15.79:0/2187182861 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400b870 con 0x7f2aa815aa10 2023-06-08T03:51:33.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa77fe700 1 -- 172.21.15.79:0/2187182861 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9400b9d0 con 0x7f2aa815aa10 2023-06-08T03:51:33.420 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aa77fe700 1 -- 172.21.15.79:0/2187182861 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9400bd60 con 0x7f2aa815aa10 2023-06-08T03:51:33.422 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aaec48700 1 -- 172.21.15.79:0/2187182861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa815aa10 msgr2=0x7f2aa80ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.414+0000 7f2aaec48700 1 --2- 172.21.15.79:0/2187182861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa815aa10 0x7f2aa80ce3f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80b33b0 tx=0x7f2a94009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.415+0000 7f2aaec48700 1 -- 172.21.15.79:0/2187182861 shutdown_connections 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.415+0000 7f2aaec48700 1 --2- 172.21.15.79:0/2187182861 >> v2:172.21.15.156:3300/0 conn(0x7f2aa815a640 0x7f2aa80b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.415+0000 7f2aaec48700 1 --2- 172.21.15.79:0/2187182861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa815aa10 0x7f2aa80ce3f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.415+0000 7f2aaec48700 1 -- 172.21.15.79:0/2187182861 >> 172.21.15.79:0/2187182861 conn(0x7f2aa806a670 msgr2=0x7f2aa8099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.415+0000 7f2aaec48700 1 -- 172.21.15.79:0/2187182861 shutdown_connections 2023-06-08T03:51:33.423 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.415+0000 7f2aaec48700 1 -- 172.21.15.79:0/2187182861 wait complete. 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.416+0000 7f2aaec48700 1 Processor -- start 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.416+0000 7f2aaec48700 1 -- start start 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 0x7f2aa80ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2aa815a640 0x7f2aa80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2aa815aa10 0x7f2aa8155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa8156070 con 0x7f2aa80ce930 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa815b880 con 0x7f2aa815a640 2023-06-08T03:51:33.424 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa815bb60 con 0x7f2aa815aa10 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 0x7f2aa80ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 0x7f2aa80ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37018/0 (socket says 172.21.15.79:37018) 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 -- 172.21.15.79:0/3976554842 learned_addr learned my addr 172.21.15.79:0/3976554842 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 -- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2aa815aa10 msgr2=0x7f2aa8155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2aa815aa10 0x7f2aa8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 -- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2aa815a640 msgr2=0x7f2aa80c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.425 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2aa815a640 0x7f2aa80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 -- 172.21.15.79:0/3976554842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94009550 con 0x7f2aa80ce930 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aac9e4700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 0x7f2aa80ca620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e810 tx=0x7f2a9c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004230 con 0x7f2aa80ce930 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c004390 con 0x7f2aa80ce930 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa81562e0 con 0x7f2aa80ce930 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0036a0 con 0x7f2aa80ce930 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.417+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa80c7e30 con 0x7f2aa80ce930 2023-06-08T03:51:33.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.418+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa8156520 con 0x7f2aa80ce930 2023-06-08T03:51:33.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.419+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a9c003800 con 0x7f2aa80ce930 2023-06-08T03:51:33.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.419+0000 7f2aa57fa700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a8c06ace0 0x7f2a8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.427 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.419+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a9c0891f0 con 0x7f2aa80ce930 2023-06-08T03:51:33.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.419+0000 7f2aa7fff700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a8c06ace0 0x7f2a8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.420+0000 7f2aa7fff700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a8c06ace0 0x7f2a8c06d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8099fb0 tx=0x7f2a94010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.423+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a9c054040 con 0x7f2aa80ce930 2023-06-08T03:51:33.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.649+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2aa80d1260 con 0x7f2aa80ce930 2023-06-08T03:51:33.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.650+0000 7f2aa57fa700 1 -- 172.21.15.79:0/3976554842 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f2a9c057660 con 0x7f2aa80ce930 2023-06-08T03:51:33.654 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:33.655 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:51:33.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a8c06ace0 msgr2=0x7f2a8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a8c06ace0 0x7f2a8c06d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8099fb0 tx=0x7f2a94010000 comp rx=0 tx=0).stop 2023-06-08T03:51:33.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 msgr2=0x7f2aa80ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 0x7f2aa80ca620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00e810 tx=0x7f2a9c00ebd0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 shutdown_connections 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f2a8c06ace0 0x7f2a8c06d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2aa815aa10 0x7f2aa8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa80ce930 0x7f2aa80ca620 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 --2- 172.21.15.79:0/3976554842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2aa815a640 0x7f2aa80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 >> 172.21.15.79:0/3976554842 conn(0x7f2aa806a670 msgr2=0x7f2aa80d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 shutdown_connections 2023-06-08T03:51:33.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.652+0000 7f2aaec48700 1 -- 172.21.15.79:0/3976554842 wait complete. 2023-06-08T03:51:33.658 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:51:33.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:33.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.784+0000 7f4fa1c9e700 1 Processor -- start 2023-06-08T03:51:33.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.784+0000 7f4fa1c9e700 1 -- start start 2023-06-08T03:51:33.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4fa1c9e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4f9c06a460 0x7f4f9c071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4fa1c9e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c0cee20 con 0x7f4f9c15f6c0 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4fa1c9e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c06fa00 con 0x7f4f9c0cd6c0 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4fa1c9e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c0cf330 con 0x7f4f9c06a460 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9b7fe700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4f9c15f6c0 0x7f4f9c069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60502/0 (socket says 172.21.15.79:60502) 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9b7fe700 1 -- 172.21.15.79:0/2942282191 learned_addr learned my addr 172.21.15.79:0/2942282191 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384943590 0 0) 0x7f4f9c0cee20 con 0x7f4f9c15f6c0 2023-06-08T03:51:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2942282191 >> v2:172.21.15.156:3300/0 conn(0x7f4f9c06a460 0x7f4f9c071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f840035b0 con 0x7f4f9c15f6c0 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1578609546 0 0) 0x7f4f840035b0 con 0x7f4f9c15f6c0 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 >> v2:172.21.15.156:3300/0 conn(0x7f4f9c06a460 msgr2=0x7f4f9c071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 --2- 172.21.15.79:0/2942282191 >> v2:172.21.15.156:3300/0 conn(0x7f4f9c06a460 0x7f4f9c071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 >> v1:172.21.15.156:6789/0 conn(0x7f4f9c0cd6c0 legacy=0x7f4f9c071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c0cd1e0 con 0x7f4f9c15f6c0 2023-06-08T03:51:33.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f4f90001670 con 0x7f4f9c15f6c0 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 >> v1:172.21.15.79:6789/0 conn(0x7f4f9c15f6c0 legacy=0x7f4f9c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f84003a70 0x7f4f84003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f84004850 0x7f4f84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84005200 0x7f4f8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0cf330 con 0x7f4f84004850 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8400c0e0 con 0x7f4f84005200 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.785+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8400c2a0 con 0x7f4f84003a70 2023-06-08T03:51:33.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f84004850 0x7f4f84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9b7fe700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f84003a70 0x7f4f84003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9bfff700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84005200 0x7f4f8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 -- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f84003a70 msgr2=0x7f4f84003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f84003a70 0x7f4f84003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 -- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84005200 msgr2=0x7f4f8400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84005200 0x7f4f8400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 -- 172.21.15.79:0/2942282191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840035b0 con 0x7f4f84004850 2023-06-08T03:51:33.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9bfff700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84005200 0x7f4f8400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9bfff700 1 -- 172.21.15.79:0/2942282191 reap_dead start 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f84004850 0x7f4f84004cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009b60 tx=0x7f4f84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee5901507181f2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00e6c0 con 0x7f4f84004850 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00e820 con 0x7f4f84004850 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4f9a7fc700 1 -- 172.21.15.79:0/2942282191 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00eb90 con 0x7f4f84004850 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2942282191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f84004850 msgr2=0x7f4f84004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.786+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f84004850 0x7f4f84004cc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009b60 tx=0x7f4f84005880 comp rx=0 tx=0).stop 2023-06-08T03:51:33.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.787+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2942282191 shutdown_connections 2023-06-08T03:51:33.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.787+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2942282191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f84004850 0x7f4f84004cc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.787+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2942282191 >> 172.21.15.79:0/2942282191 conn(0x7f4f9c0b1d20 msgr2=0x7f4f9c06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:33.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.787+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2942282191 shutdown_connections 2023-06-08T03:51:33.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.787+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2942282191 wait complete. 2023-06-08T03:51:33.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.788+0000 7f4fa1c9e700 1 Processor -- start 2023-06-08T03:51:33.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.788+0000 7f4fa1c9e700 1 -- start start 2023-06-08T03:51:33.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84004850 0x7f4f9c1f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 0x7f4f9c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f9c1e74b0 0x7f4f9c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1e6410 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1e66f0 con 0x7f4f84004850 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1e69d0 con 0x7f4f9c1e74b0 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 0x7f4f9c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 0x7f4f9c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37040/0 (socket says 172.21.15.79:37040) 2023-06-08T03:51:33.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 -- 172.21.15.79:0/2975620702 learned_addr learned my addr 172.21.15.79:0/2975620702 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9b7fe700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84004850 0x7f4f9c1f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9bfff700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f9c1e74b0 0x7f4f9c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 -- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f9c1e74b0 msgr2=0x7f4f9c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f9c1e74b0 0x7f4f9c14f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 -- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84004850 msgr2=0x7f4f9c1f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84004850 0x7f4f9c1f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 -- 172.21.15.79:0/2975620702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c1e6cb0 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f9affd700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 0x7f4f9c06ed60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005830 tx=0x7f4f8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00e6c0 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c003ad0 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c0045d0 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c1e6f90 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.789+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c06f2a0 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.790+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c1e7150 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.791+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4f8c013470 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.791+0000 7f4f83fff700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8806ace0 0x7f4f8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:33.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.791+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4f8c08afe0 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.791+0000 7f4f9b7fe700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8806ace0 0x7f4f8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:33.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.792+0000 7f4f9b7fe700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8806ace0 0x7f4f8806d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f90007df0 tx=0x7f4f90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:33.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.795+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4f8c055f50 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.983+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f9c073e90 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.985+0000 7f4f83fff700 1 -- 172.21.15.79:0/2975620702 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f4f8c059570 con 0x7f4f9c1e5c00 2023-06-08T03:51:33.989 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:33.989 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:33.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8806ace0 msgr2=0x7f4f8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8806ace0 0x7f4f8806d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f90007df0 tx=0x7f4f90009040 comp rx=0 tx=0).stop 2023-06-08T03:51:33.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 msgr2=0x7f4f9c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:33.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 0x7f4f9c06ed60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005830 tx=0x7f4f8c0047c0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 shutdown_connections 2023-06-08T03:51:33.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f4f8806ace0 0x7f4f8806d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4f9c1e74b0 0x7f4f9c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f84004850 0x7f4f9c1f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 --2- 172.21.15.79:0/2975620702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f9c1e5c00 0x7f4f9c06ed60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:33.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 >> 172.21.15.79:0/2975620702 conn(0x7f4f9c0b1d20 msgr2=0x7f4f9c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:33.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 shutdown_connections 2023-06-08T03:51:33.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:33.987+0000 7f4fa1c9e700 1 -- 172.21.15.79:0/2975620702 wait complete. 2023-06-08T03:51:34.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-08T03:50:22.311526+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-08T03:50:23.630403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-06-08T03:50:26.911508+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-06-08T03:51:34.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:34.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.122+0000 7f876dd84700 1 Processor -- start 2023-06-08T03:51:34.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.122+0000 7f876dd84700 1 -- start start 2023-06-08T03:51:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f876dd84700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8768158d50 0x7f87680ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f876dd84700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876815d3d0 0x7f87680cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f876dd84700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f87680cc4c0 0x7f876806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f876dd84700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87680cc920 con 0x7f87680cc4c0 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f876dd84700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87680cd7b0 con 0x7f8768158d50 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f876dd84700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87680cda50 con 0x7f876815d3d0 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f87680cc4c0 0x7f876806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f87680cc4c0 0x7f876806eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37050/0 (socket says 172.21.15.79:37050) 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 -- 172.21.15.79:0/1606487083 learned_addr learned my addr 172.21.15.79:0/1606487083 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8766ffd700 1 --2- 172.21.15.79:0/1606487083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876815d3d0 0x7f87680cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f87677fe700 1 --2- 172.21.15.79:0/1606487083 >> v2:172.21.15.156:3300/0 conn(0x7f8768158d50 0x7f87680ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 -- 172.21.15.79:0/1606487083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876815d3d0 msgr2=0x7f87680cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 --2- 172.21.15.79:0/1606487083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876815d3d0 0x7f87680cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 -- 172.21.15.79:0/1606487083 >> v2:172.21.15.156:3300/0 conn(0x7f8768158d50 msgr2=0x7f87680ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 --2- 172.21.15.79:0/1606487083 >> v2:172.21.15.156:3300/0 conn(0x7f8768158d50 0x7f87680ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 -- 172.21.15.79:0/1606487083 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87680cdd30 con 0x7f87680cc4c0 2023-06-08T03:51:34.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f87677fe700 1 --2- 172.21.15.79:0/1606487083 >> v2:172.21.15.156:3300/0 conn(0x7f8768158d50 0x7f87680ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.123+0000 7f8767fff700 1 --2- 172.21.15.79:0/1606487083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f876806eac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f876806e5d0 tx=0x7f876000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edea3f592424a0e7 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f87667fc700 1 -- 172.21.15.79:0/1606487083 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760010070 con 0x7f87680cc4c0 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f87667fc700 1 -- 172.21.15.79:0/1606487083 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876000dce0 con 0x7f87680cc4c0 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f87667fc700 1 -- 172.21.15.79:0/1606487083 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000b310 con 0x7f87680cc4c0 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 -- 172.21.15.79:0/1606487083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 msgr2=0x7f876806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 --2- 172.21.15.79:0/1606487083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f876806eac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f876806e5d0 tx=0x7f876000d430 comp rx=0 tx=0).stop 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 -- 172.21.15.79:0/1606487083 shutdown_connections 2023-06-08T03:51:34.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 --2- 172.21.15.79:0/1606487083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f876806eac0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 --2- 172.21.15.79:0/1606487083 >> v2:172.21.15.156:3300/0 conn(0x7f8768158d50 0x7f87680ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 --2- 172.21.15.79:0/1606487083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f876815d3d0 0x7f87680cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 -- 172.21.15.79:0/1606487083 >> 172.21.15.79:0/1606487083 conn(0x7f87680b1d30 msgr2=0x7f876806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:34.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 -- 172.21.15.79:0/1606487083 shutdown_connections 2023-06-08T03:51:34.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.124+0000 7f876dd84700 1 -- 172.21.15.79:0/1606487083 wait complete. 2023-06-08T03:51:34.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.125+0000 7f876dd84700 1 Processor -- start 2023-06-08T03:51:34.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.125+0000 7f876dd84700 1 -- start start 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f876dd84700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f87680c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f876dd84700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8768158d50 0x7f87680c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f876dd84700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876815d3d0 0x7f87680b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f876dd84700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87680c53f0 con 0x7f87680cc4c0 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f876dd84700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87680b3e10 con 0x7f876815d3d0 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f876dd84700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8768152da0 con 0x7f8768158d50 2023-06-08T03:51:34.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f87677fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f87680c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f87677fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f87680c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37052/0 (socket says 172.21.15.79:37052) 2023-06-08T03:51:34.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f87677fe700 1 -- 172.21.15.79:0/3475260451 learned_addr learned my addr 172.21.15.79:0/3475260451 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:34.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f8767fff700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876815d3d0 0x7f87680b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.126+0000 7f8766ffd700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8768158d50 0x7f87680c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f87677fe700 1 -- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8768158d50 msgr2=0x7f87680c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f87677fe700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8768158d50 0x7f87680c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f87677fe700 1 -- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876815d3d0 msgr2=0x7f87680b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f87677fe700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876815d3d0 0x7f87680b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f87677fe700 1 -- 172.21.15.79:0/3475260451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876000bd40 con 0x7f87680cc4c0 2023-06-08T03:51:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f8766ffd700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8768158d50 0x7f87680c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:34.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f87677fe700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f87680c4190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f875c00d800 tx=0x7f875c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c004030 con 0x7f87680cc4c0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c004190 con 0x7f87680cc4c0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87680c4770 con 0x7f87680cc4c0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c004520 con 0x7f87680cc4c0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.127+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87680c4d30 con 0x7f87680cc4c0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.128+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87680c5660 con 0x7f87680cc4c0 2023-06-08T03:51:34.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.128+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f875c0036a0 con 0x7f87680cc4c0 2023-06-08T03:51:34.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.129+0000 7f874ffff700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f875406ace0 0x7f875406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.129+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f875c089b70 con 0x7f87680cc4c0 2023-06-08T03:51:34.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.129+0000 7f8766ffd700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f875406ace0 0x7f875406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.130+0000 7f8766ffd700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f875406ace0 0x7f875406d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8768071600 tx=0x7f8758008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.139 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.133+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f875c058450 con 0x7f87680cc4c0 2023-06-08T03:51:34.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.321+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8768073e90 con 0x7f87680cc4c0 2023-06-08T03:51:34.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.321+0000 7f874ffff700 1 -- 172.21.15.79:0/3475260451 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f875c057fe0 con 0x7f87680cc4c0 2023-06-08T03:51:34.326 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:34.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f875406ace0 msgr2=0x7f875406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f875406ace0 0x7f875406d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8768071600 tx=0x7f8758008040 comp rx=0 tx=0).stop 2023-06-08T03:51:34.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 msgr2=0x7f87680c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f87680c4190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f875c00d800 tx=0x7f875c00db10 comp rx=0 tx=0).stop 2023-06-08T03:51:34.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 shutdown_connections 2023-06-08T03:51:34.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f876815d3d0 0x7f87680b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87680cc4c0 0x7f87680c4190 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f875406ace0 0x7f875406d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 --2- 172.21.15.79:0/3475260451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8768158d50 0x7f87680c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 >> 172.21.15.79:0/3475260451 conn(0x7f87680b1d30 msgr2=0x7f87680af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:34.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 shutdown_connections 2023-06-08T03:51:34.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.323+0000 7f876dd84700 1 -- 172.21.15.79:0/3475260451 wait complete. 2023-06-08T03:51:34.339 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:34.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.457+0000 7f843a0c1700 1 Processor -- start 2023-06-08T03:51:34.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.458+0000 7f843a0c1700 1 -- start start 2023-06-08T03:51:34.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.458+0000 7f843a0c1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f84340cd080 0x7f84340cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.458+0000 7f843a0c1700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84340cae50 con 0x7f84340ce480 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.458+0000 7f843a0c1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84340afee0 con 0x7f84340d0060 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.458+0000 7f843a0c1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84340cbea0 con 0x7f84340cd080 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8433fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f84340ce480 0x7f84340ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60536/0 (socket says 172.21.15.79:60536) 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8433fff700 1 -- 172.21.15.79:0/1213346284 learned_addr learned my addr 172.21.15.79:0/1213346284 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 --2- 172.21.15.79:0/1213346284 >> v2:172.21.15.156:3300/0 conn(0x7f84340cd080 0x7f84340cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962393744 0 0) 0x7f84340cae50 con 0x7f84340ce480 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f841c0035b0 con 0x7f84340ce480 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166334259 0 0) 0x7f841c0035b0 con 0x7f84340ce480 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 >> v2:172.21.15.156:3300/0 conn(0x7f84340cd080 msgr2=0x7f84340cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 --2- 172.21.15.79:0/1213346284 >> v2:172.21.15.156:3300/0 conn(0x7f84340cd080 0x7f84340cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 >> v1:172.21.15.156:6789/0 conn(0x7f84340d0060 legacy=0x7f84340af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843415d930 con 0x7f84340ce480 2023-06-08T03:51:34.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f842c0019f0 con 0x7f84340ce480 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 >> v1:172.21.15.79:6789/0 conn(0x7f84340ce480 legacy=0x7f84340ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841c003a00 0x7f841c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f841c004850 0x7f841c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c005200 0x7f841c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c00c050 con 0x7f841c004850 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c00c1c0 con 0x7f841c005200 2023-06-08T03:51:34.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c00c380 con 0x7f841c003a00 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f841c004850 0x7f841c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8433fff700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c005200 0x7f841c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f84337fe700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841c003a00 0x7f841c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 -- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841c003a00 msgr2=0x7f841c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f841c003a00 0x7f841c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 -- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c005200 msgr2=0x7f841c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:34.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c005200 0x7f841c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8432ffd700 1 -- 172.21.15.79:0/1213346284 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c0035b0 con 0x7f841c004850 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.459+0000 7f8433fff700 1 -- 172.21.15.79:0/1213346284 reap_dead start 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f8432ffd700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f841c004850 0x7f841c004cc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8424009b30 tx=0x7f841c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38819671a5729037 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400e6c0 con 0x7f841c004850 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842400e820 con 0x7f841c004850 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f84327fc700 1 -- 172.21.15.79:0/1213346284 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400eb90 con 0x7f841c004850 2023-06-08T03:51:34.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 -- 172.21.15.79:0/1213346284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f841c004850 msgr2=0x7f841c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f841c004850 0x7f841c004cc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8424009b30 tx=0x7f841c005880 comp rx=0 tx=0).stop 2023-06-08T03:51:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 -- 172.21.15.79:0/1213346284 shutdown_connections 2023-06-08T03:51:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 --2- 172.21.15.79:0/1213346284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f841c004850 0x7f841c004cc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 -- 172.21.15.79:0/1213346284 >> 172.21.15.79:0/1213346284 conn(0x7f843406e1e0 msgr2=0x7f843406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 -- 172.21.15.79:0/1213346284 shutdown_connections 2023-06-08T03:51:34.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.460+0000 7f843a0c1700 1 -- 172.21.15.79:0/1213346284 wait complete. 2023-06-08T03:51:34.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.461+0000 7f843a0c1700 1 Processor -- start 2023-06-08T03:51:34.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 -- start start 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c004850 0x7f84340c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84341f3e30 0x7f84340b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 0x7f843414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84340c42b0 con 0x7f84341f3490 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84340c4590 con 0x7f841c004850 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f843a0c1700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84340c4870 con 0x7f84341f3e30 2023-06-08T03:51:34.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.462+0000 7f8433fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 0x7f843414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 0x7f843414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37076/0 (socket says 172.21.15.79:37076) 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 -- 172.21.15.79:0/2887840765 learned_addr learned my addr 172.21.15.79:0/2887840765 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f84337fe700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c004850 0x7f84340c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8432ffd700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84341f3e30 0x7f84340b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 -- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84341f3e30 msgr2=0x7f84340b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84341f3e30 0x7f84340b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 -- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c004850 msgr2=0x7f84340c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c004850 0x7f84340c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 -- 172.21.15.79:0/2887840765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340c4b50 con 0x7f84341f3490 2023-06-08T03:51:34.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f84337fe700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c004850 0x7f84340c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:34.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f8433fff700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 0x7f843414b6b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8434072150 tx=0x7f842c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00ed20 con 0x7f84341f3490 2023-06-08T03:51:34.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c00ee80 con 0x7f84341f3490 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84340c4e30 con 0x7f84341f3490 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c013660 con 0x7f84341f3490 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.463+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84340b0610 con 0x7f84341f3490 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.464+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84340c5020 con 0x7f84341f3490 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.464+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f842c0137c0 con 0x7f84341f3490 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.465+0000 7f841bfff700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f842006ace0 0x7f842006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.465+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f842c089ce0 con 0x7f84341f3490 2023-06-08T03:51:34.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.465+0000 7f84337fe700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f842006ace0 0x7f842006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.465+0000 7f84337fe700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f842006ace0 0x7f842006d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8428000c00 tx=0x7f8428009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.469+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f842c054b30 con 0x7f84341f3490 2023-06-08T03:51:34.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.652+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8434073e90 con 0x7f84341f3490 2023-06-08T03:51:34.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.652+0000 7f841bfff700 1 -- 172.21.15.79:0/2887840765 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f842c058150 con 0x7f84341f3490 2023-06-08T03:51:34.656 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:34.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f842006ace0 msgr2=0x7f842006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f842006ace0 0x7f842006d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8428000c00 tx=0x7f8428009040 comp rx=0 tx=0).stop 2023-06-08T03:51:34.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 msgr2=0x7f843414b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 0x7f843414b6b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8434072150 tx=0x7f842c007c90 comp rx=0 tx=0).stop 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 shutdown_connections 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84341f3490 0x7f843414b6b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f841c004850 0x7f84340c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f842006ace0 0x7f842006d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 --2- 172.21.15.79:0/2887840765 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f84341f3e30 0x7f84340b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 >> 172.21.15.79:0/2887840765 conn(0x7f843406e1e0 msgr2=0x7f84340b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:34.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 shutdown_connections 2023-06-08T03:51:34.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.654+0000 7f843a0c1700 1 -- 172.21.15.79:0/2887840765 wait complete. 2023-06-08T03:51:34.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-08T03:51:34.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.789+0000 7f79a3aef700 1 Processor -- start 2023-06-08T03:51:34.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.789+0000 7f79a3aef700 1 -- start start 2023-06-08T03:51:34.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a3aef700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f799c070ee0 0x7f799c06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a3aef700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799c06da90 con 0x7f799c158780 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a3aef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799c0ce690 con 0x7f799c0ccb30 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a3aef700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799c06e110 con 0x7f799c070ee0 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a188b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f799c158780 0x7f799c0c9380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60556/0 (socket says 172.21.15.79:60556) 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a188b700 1 -- 172.21.15.79:0/3288766651 learned_addr learned my addr 172.21.15.79:0/3288766651 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a208c700 1 --2- 172.21.15.79:0/3288766651 >> v2:172.21.15.156:3300/0 conn(0x7f799c070ee0 0x7f799c06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346408168 0 0) 0x7f799c06da90 con 0x7f799c158780 2023-06-08T03:51:34.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79880035b0 con 0x7f799c158780 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.790+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836002740 0 0) 0x7f799c0ce690 con 0x7f799c0ccb30 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c06da90 con 0x7f799c0ccb30 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a208c700 1 -- 172.21.15.79:0/3288766651 >> v1:172.21.15.156:6789/0 conn(0x7f799c0ccb30 legacy=0x7f799c0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a208c700 1 -- 172.21.15.79:0/3288766651 >> v1:172.21.15.79:6789/0 conn(0x7f799c158780 legacy=0x7f799c0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 138553540 0 0) 0x7f79880035b0 con 0x7f799c158780 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a208c700 1 -- 172.21.15.79:0/3288766651 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c0ce970 con 0x7f799c070ee0 2023-06-08T03:51:34.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a208c700 1 --2- 172.21.15.79:0/3288766651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c06b230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f799400aea0 tx=0x7f799400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e9eaae28b69114 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400f070 con 0x7f799c070ee0 2023-06-08T03:51:34.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994003a20 con 0x7f799c070ee0 2023-06-08T03:51:34.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a0889700 1 -- 172.21.15.79:0/3288766651 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998001670 con 0x7f799c070ee0 2023-06-08T03:51:34.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a3aef700 1 -- 172.21.15.79:0/3288766651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 msgr2=0x7f799c06b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a3aef700 1 --2- 172.21.15.79:0/3288766651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c06b230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f799400aea0 tx=0x7f799400c330 comp rx=0 tx=0).stop 2023-06-08T03:51:34.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.791+0000 7f79a3aef700 1 -- 172.21.15.79:0/3288766651 shutdown_connections 2023-06-08T03:51:34.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.792+0000 7f79a3aef700 1 --2- 172.21.15.79:0/3288766651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c06b230 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.792+0000 7f79a3aef700 1 -- 172.21.15.79:0/3288766651 >> 172.21.15.79:0/3288766651 conn(0x7f799c0ced30 msgr2=0x7f799c0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:34.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.792+0000 7f79a3aef700 1 -- 172.21.15.79:0/3288766651 shutdown_connections 2023-06-08T03:51:34.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.792+0000 7f79a3aef700 1 -- 172.21.15.79:0/3288766651 wait complete. 2023-06-08T03:51:34.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.793+0000 7f79a3aef700 1 Processor -- start 2023-06-08T03:51:34.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.793+0000 7f79a3aef700 1 -- start start 2023-06-08T03:51:34.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a3aef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a3aef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c0ccb30 0x7f799c1ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a3aef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 0x7f799c1edf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a3aef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0c4e30 con 0x7f799c158780 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a3aef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0c41b0 con 0x7f799c070ee0 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a3aef700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c0c4490 con 0x7f799c0ccb30 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 0x7f799c1edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 0x7f799c1edf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37090/0 (socket says 172.21.15.79:37090) 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 -- 172.21.15.79:0/1902394624 learned_addr learned my addr 172.21.15.79:0/1902394624 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a188b700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a108a700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c0ccb30 0x7f799c1ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 -- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c0ccb30 msgr2=0x7f799c1ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c0ccb30 0x7f799c1ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 -- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 msgr2=0x7f799c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.794+0000 7f79a208c700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c0b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79a208c700 1 -- 172.21.15.79:0/1902394624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799400ab80 con 0x7f799c158780 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79a208c700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 0x7f799c1edf20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f799400c370 tx=0x7f7994004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994004500 con 0x7f799c158780 2023-06-08T03:51:34.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994004660 con 0x7f799c158780 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c0c4770 con 0x7f799c158780 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994008440 con 0x7f799c158780 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.795+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c1564a0 con 0x7f799c158780 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.796+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c06f1d0 con 0x7f799c158780 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.796+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79940085a0 con 0x7f799c158780 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.797+0000 7f79927fc700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f798006ace0 0x7f798006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:34.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.797+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f799408c3e0 con 0x7f799c158780 2023-06-08T03:51:34.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.797+0000 7f79a188b700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f798006ace0 0x7f798006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:34.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.798+0000 7f79a188b700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f798006ace0 0x7f798006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7998000f60 tx=0x7f79980014b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:34.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.801+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7994057290 con 0x7f799c158780 2023-06-08T03:51:34.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.986+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f799c073e90 con 0x7f799c158780 2023-06-08T03:51:34.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.987+0000 7f79927fc700 1 -- 172.21.15.79:0/1902394624 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f799405a8b0 con 0x7f799c158780 2023-06-08T03:51:34.991 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f798006ace0 msgr2=0x7f798006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f798006ace0 0x7f798006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7998000f60 tx=0x7f79980014b0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 msgr2=0x7f799c1edf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 0x7f799c1edf20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f799400c370 tx=0x7f7994004ad0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 shutdown_connections 2023-06-08T03:51:34.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f799c158780 0x7f799c1edf20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f799c070ee0 0x7f799c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f798006ace0 0x7f798006d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 --2- 172.21.15.79:0/1902394624 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f799c0ccb30 0x7f799c1ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:34.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 >> 172.21.15.79:0/1902394624 conn(0x7f799c0ced30 msgr2=0x7f799c0ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:34.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 shutdown_connections 2023-06-08T03:51:34.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:34.989+0000 7f79a3aef700 1 -- 172.21.15.79:0/1902394624 wait complete. 2023-06-08T03:51:35.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-08T03:51:35.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.122+0000 7f80898f1700 1 Processor -- start 2023-06-08T03:51:35.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.122+0000 7f80898f1700 1 -- start start 2023-06-08T03:51:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80898f1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80840d0b80 0x7f80840d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80898f1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80840ceb40 0x7f80840af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80898f1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cdaa0 con 0x7f808415c280 2023-06-08T03:51:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80898f1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840cdd80 con 0x7f80840d0b80 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80898f1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840ce060 con 0x7f80840ceb40 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80840d0b80 0x7f80840d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f80840d0b80 0x7f80840d23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37094/0 (socket says 172.21.15.79:37094) 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 -- 172.21.15.79:0/3028278501 learned_addr learned my addr 172.21.15.79:0/3028278501 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80837fe700 1 --2- 172.21.15.79:0/3028278501 >> v2:172.21.15.156:3300/0 conn(0x7f80840ceb40 0x7f80840af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 -- 172.21.15.79:0/3028278501 >> v2:172.21.15.156:3300/0 conn(0x7f80840ceb40 msgr2=0x7f80840af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 --2- 172.21.15.79:0/3028278501 >> v2:172.21.15.156:3300/0 conn(0x7f80840ceb40 0x7f80840af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 -- 172.21.15.79:0/3028278501 >> v1:172.21.15.156:6789/0 conn(0x7f808415c280 legacy=0x7f80840d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 -- 172.21.15.79:0/3028278501 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840ce340 con 0x7f80840d0b80 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.123+0000 7f80827fc700 1 --2- 172.21.15.79:0/3028278501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80840d0b80 0x7f80840d23f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f808408bb90 tx=0x7f8074009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50d703e56558689 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f8081ffb700 1 -- 172.21.15.79:0/3028278501 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400b870 con 0x7f80840d0b80 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f8081ffb700 1 -- 172.21.15.79:0/3028278501 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807400b9d0 con 0x7f80840d0b80 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f8081ffb700 1 -- 172.21.15.79:0/3028278501 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400bd60 con 0x7f80840d0b80 2023-06-08T03:51:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 -- 172.21.15.79:0/3028278501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80840d0b80 msgr2=0x7f80840d23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 --2- 172.21.15.79:0/3028278501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80840d0b80 0x7f80840d23f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f808408bb90 tx=0x7f8074009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 -- 172.21.15.79:0/3028278501 shutdown_connections 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 --2- 172.21.15.79:0/3028278501 >> v2:172.21.15.156:3300/0 conn(0x7f80840ceb40 0x7f80840af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 --2- 172.21.15.79:0/3028278501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f80840d0b80 0x7f80840d23f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 -- 172.21.15.79:0/3028278501 >> 172.21.15.79:0/3028278501 conn(0x7f808409bc40 msgr2=0x7f80840d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 -- 172.21.15.79:0/3028278501 shutdown_connections 2023-06-08T03:51:35.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.124+0000 7f80898f1700 1 -- 172.21.15.79:0/3028278501 wait complete. 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.125+0000 7f80898f1700 1 Processor -- start 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 -- start start 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80840ceb40 0x7f8084156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80840d0b80 0x7f8084156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 0x7f80840b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cb790 con 0x7f808415c280 2023-06-08T03:51:35.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084156760 con 0x7f80840ceb40 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80898f1700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cab10 con 0x7f80840d0b80 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80837fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 0x7f80840b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80837fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 0x7f80840b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37102/0 (socket says 172.21.15.79:37102) 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80837fe700 1 -- 172.21.15.79:0/381352911 learned_addr learned my addr 172.21.15.79:0/381352911 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f8082ffd700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80840ceb40 0x7f8084156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.126+0000 7f80827fc700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80840d0b80 0x7f8084156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80837fe700 1 -- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80840d0b80 msgr2=0x7f8084156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80837fe700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80840d0b80 0x7f8084156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80837fe700 1 -- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80840ceb40 msgr2=0x7f8084156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80837fe700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80840ceb40 0x7f8084156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80837fe700 1 -- 172.21.15.79:0/381352911 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074009550 con 0x7f808415c280 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80827fc700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80840d0b80 0x7f8084156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f8082ffd700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80840ceb40 0x7f8084156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80837fe700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 0x7f80840b5760 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f807c007e30 tx=0x7f807c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.136 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00ee40 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c014420 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80840cadf0 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0147d0 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.127+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80840cb3b0 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.128+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80840cb8d0 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.128+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f807c014930 con 0x7f808415c280 2023-06-08T03:51:35.137 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.129+0000 7f806bfff700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f807006ace0 0x7f807006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.129+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f807c089ec0 con 0x7f808415c280 2023-06-08T03:51:35.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.129+0000 7f8082ffd700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f807006ace0 0x7f807006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.130+0000 7f8082ffd700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f807006ace0 0x7f807006d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8078000fb0 tx=0x7f8078009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.138 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.133+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f80840cb8d0 con 0x7f808415c280 2023-06-08T03:51:35.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.320+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80840cb8d0 con 0x7f808415c280 2023-06-08T03:51:35.324 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.320+0000 7f806bfff700 1 -- 172.21.15.79:0/381352911 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80840cb8d0 con 0x7f808415c280 2023-06-08T03:51:35.324 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2023-06-08T03:51:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f807006ace0 msgr2=0x7f807006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.326 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f807006ace0 0x7f807006d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8078000fb0 tx=0x7f8078009040 comp rx=0 tx=0).stop 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 msgr2=0x7f80840b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 0x7f80840b5760 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f807c007e30 tx=0x7f807c00e3c0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 shutdown_connections 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808415c280 0x7f80840b5760 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80840ceb40 0x7f8084156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f807006ace0 0x7f807006d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 --2- 172.21.15.79:0/381352911 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80840d0b80 0x7f8084156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 >> 172.21.15.79:0/381352911 conn(0x7f808409bc40 msgr2=0x7f80840af670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:35.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 shutdown_connections 2023-06-08T03:51:35.328 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.322+0000 7f80898f1700 1 -- 172.21.15.79:0/381352911 wait complete. 2023-06-08T03:51:35.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-06-08T03:51:35.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.456+0000 7f24e4bae700 1 Processor -- start 2023-06-08T03:51:35.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.456+0000 7f24e4bae700 1 -- start start 2023-06-08T03:51:35.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.457+0000 7f24e4bae700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015c1f0 0x7f24e006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.457+0000 7f24e4bae700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f24e0069d50 0x7f24e006a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.457+0000 7f24e4bae700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e00cd140 con 0x7f24e0071fe0 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.457+0000 7f24e4bae700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e006a710 con 0x7f24e0069d50 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.457+0000 7f24e4bae700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0071110 con 0x7f24e015c1f0 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24ddd9b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f24e0071fe0 0x7f24e00cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60582/0 (socket says 172.21.15.79:60582) 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24ddd9b700 1 -- 172.21.15.79:0/3641299378 learned_addr learned my addr 172.21.15.79:0/3641299378 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863248154 0 0) 0x7f24e00cd140 con 0x7f24e0071fe0 2023-06-08T03:51:35.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c80035b0 con 0x7f24e0071fe0 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24ded9d700 1 --2- 172.21.15.79:0/3641299378 >> v2:172.21.15.156:3300/0 conn(0x7f24e0069d50 0x7f24e006a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24de59c700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015c1f0 0x7f24e006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2731296159 0 0) 0x7f24c80035b0 con 0x7f24e0071fe0 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015c1f0 msgr2=0x7f24e006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015c1f0 0x7f24e006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 >> v2:172.21.15.156:3300/0 conn(0x7f24e0069d50 msgr2=0x7f24e006a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:35.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 --2- 172.21.15.79:0/3641299378 >> v2:172.21.15.156:3300/0 conn(0x7f24e0069d50 0x7f24e006a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0071be0 con 0x7f24e0071fe0 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f24d0001670 con 0x7f24e0071fe0 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 >> v1:172.21.15.79:6789/0 conn(0x7f24e0071fe0 legacy=0x7f24e00cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24c8003a00 0x7f24c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8004850 0x7f24c8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24c8005200 0x7f24c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e006a710 con 0x7f24c8005200 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0071110 con 0x7f24c8004850 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c800c0b0 con 0x7f24c8003a00 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24de59c700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015c1f0 0x7f24e006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24ded9d700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24c8005200 0x7f24c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24ddd9b700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8004850 0x7f24c8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.458+0000 7f24de59c700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24c8003a00 0x7f24c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 -- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24c8003a00 msgr2=0x7f24c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24c8003a00 0x7f24c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 -- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8004850 msgr2=0x7f24c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8004850 0x7f24c8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 -- 172.21.15.79:0/3641299378 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80035b0 con 0x7f24c8005200 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24c8005200 0x7f24c800ba10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009fc0 tx=0x7f24c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bceb39e0fa9fad03 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d800ec30 con 0x7f24c8005200 2023-06-08T03:51:35.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d800f040 con 0x7f24c8005200 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24dd59a700 1 -- 172.21.15.79:0/3641299378 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8014480 con 0x7f24c8005200 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24e4bae700 1 -- 172.21.15.79:0/3641299378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24c8005200 msgr2=0x7f24c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24e4bae700 1 --2- 172.21.15.79:0/3641299378 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24c8005200 0x7f24c800ba10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009fc0 tx=0x7f24c8005880 comp rx=0 tx=0).stop 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24ded9d700 1 -- 172.21.15.79:0/3641299378 reap_dead start 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24e4bae700 1 -- 172.21.15.79:0/3641299378 shutdown_connections 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.459+0000 7f24e4bae700 1 -- 172.21.15.79:0/3641299378 >> 172.21.15.79:0/3641299378 conn(0x7f24e00b1d30 msgr2=0x7f24e006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.460+0000 7f24e4bae700 1 -- 172.21.15.79:0/3641299378 shutdown_connections 2023-06-08T03:51:35.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.460+0000 7f24e4bae700 1 -- 172.21.15.79:0/3641299378 wait complete. 2023-06-08T03:51:35.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.461+0000 7f24e4bae700 1 Processor -- start 2023-06-08T03:51:35.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.461+0000 7f24e4bae700 1 -- start start 2023-06-08T03:51:35.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8005200 0x7f24e00c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 0x7f24e0159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015ab00 0x7f24e00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e015a380 con 0x7f24e015a5d0 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00c40a0 con 0x7f24c8005200 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00c4380 con 0x7f24e015ab00 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 0x7f24e0159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 0x7f24e0159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37120/0 (socket says 172.21.15.79:37120) 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 -- 172.21.15.79:0/833619347 learned_addr learned my addr 172.21.15.79:0/833619347 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24de59c700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8005200 0x7f24e00c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 -- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015ab00 msgr2=0x7f24e00c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015ab00 0x7f24e00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 -- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8005200 msgr2=0x7f24e00c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8005200 0x7f24e00c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 -- 172.21.15.79:0/833619347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00c80a0 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ddd9b700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 0x7f24e0159e40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24d00017a0 tx=0x7f24d0007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d0007800 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d000f8d0 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e00c8380 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d000fc60 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.462+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00c55e0 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.463+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e00c88a0 con 0x7f24e015a5d0 2023-06-08T03:51:35.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.464+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24d001d460 con 0x7f24e015a5d0 2023-06-08T03:51:35.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.464+0000 7f24ceffd700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f24c006ace0 0x7f24c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.464+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f24d0089850 con 0x7f24e015a5d0 2023-06-08T03:51:35.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.464+0000 7f24de59c700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f24c006ace0 0x7f24c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.465+0000 7f24de59c700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f24c006ace0 0x7f24c006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f24e006f780 tx=0x7f24d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.468+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f24e00c88a0 con 0x7f24e015a5d0 2023-06-08T03:51:35.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.653+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f24e00c88a0 con 0x7f24e015a5d0 2023-06-08T03:51:35.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.653+0000 7f24ceffd700 1 -- 172.21.15.79:0/833619347 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f24e00c88a0 con 0x7f24e015a5d0 2023-06-08T03:51:35.657 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2023-06-08T03:51:35.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f24c006ace0 msgr2=0x7f24c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f24c006ace0 0x7f24c006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f24e006f780 tx=0x7f24d4008040 comp rx=0 tx=0).stop 2023-06-08T03:51:35.659 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 msgr2=0x7f24e0159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 0x7f24e0159e40 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f24d00017a0 tx=0x7f24d0007620 comp rx=0 tx=0).stop 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 shutdown_connections 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f24c006ace0 0x7f24c006d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24e015ab00 0x7f24e00c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24c8005200 0x7f24e00c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 --2- 172.21.15.79:0/833619347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f24e015a5d0 0x7f24e0159e40 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.660 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 >> 172.21.15.79:0/833619347 conn(0x7f24e00b1d30 msgr2=0x7f24e00b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 shutdown_connections 2023-06-08T03:51:35.661 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.655+0000 7f24e4bae700 1 -- 172.21.15.79:0/833619347 wait complete. 2023-06-08T03:51:35.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:51:35.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.786+0000 7fdd9a703700 1 Processor -- start 2023-06-08T03:51:35.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.786+0000 7fdd9a703700 1 -- start start 2023-06-08T03:51:35.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd9a703700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9415f6c0 0x7fdd94069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd9a703700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd940cd6c0 0x7fdd9406a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd9a703700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940cee20 con 0x7fdd9406f210 2023-06-08T03:51:35.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd9a703700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd940cf330 con 0x7fdd940cd6c0 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd9a703700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9406fa00 con 0x7fdd9415f6c0 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd937fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd940cd6c0 0x7fdd9406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd98ca0700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdd9406f210 0x7fdd940cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60604/0 (socket says 172.21.15.79:60604) 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd98ca0700 1 -- 172.21.15.79:0/2013693767 learned_addr learned my addr 172.21.15.79:0/2013693767 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd93fff700 1 --2- 172.21.15.79:0/2013693767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9415f6c0 0x7fdd94069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd937fe700 1 -- 172.21.15.79:0/2013693767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9415f6c0 msgr2=0x7fdd94069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:35.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd937fe700 1 --2- 172.21.15.79:0/2013693767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9415f6c0 0x7fdd94069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd937fe700 1 -- 172.21.15.79:0/2013693767 >> v1:172.21.15.79:6789/0 conn(0x7fdd9406f210 legacy=0x7fdd940cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd937fe700 1 -- 172.21.15.79:0/2013693767 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940cd1e0 con 0x7fdd940cd6c0 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd92ffd700 1 -- 172.21.15.79:0/2013693767 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341408296 0 0) 0x7fdd940cee20 con 0x7fdd9406f210 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.787+0000 7fdd937fe700 1 --2- 172.21.15.79:0/2013693767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd9406a290 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009840 tx=0x7fdd84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8584c5918d25ed server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd92ffd700 1 -- 172.21.15.79:0/2013693767 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84011830 con 0x7fdd940cd6c0 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd92ffd700 1 -- 172.21.15.79:0/2013693767 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84011990 con 0x7fdd940cd6c0 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd92ffd700 1 -- 172.21.15.79:0/2013693767 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84011d20 con 0x7fdd940cd6c0 2023-06-08T03:51:35.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 -- 172.21.15.79:0/2013693767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 msgr2=0x7fdd9406a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 --2- 172.21.15.79:0/2013693767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd9406a290 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009840 tx=0x7fdd84009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 -- 172.21.15.79:0/2013693767 shutdown_connections 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 --2- 172.21.15.79:0/2013693767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9415f6c0 0x7fdd94069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 --2- 172.21.15.79:0/2013693767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd9406a290 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 -- 172.21.15.79:0/2013693767 >> 172.21.15.79:0/2013693767 conn(0x7fdd940b1d20 msgr2=0x7fdd9406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 -- 172.21.15.79:0/2013693767 shutdown_connections 2023-06-08T03:51:35.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.788+0000 7fdd9a703700 1 -- 172.21.15.79:0/2013693767 wait complete. 2023-06-08T03:51:35.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.789+0000 7fdd9a703700 1 Processor -- start 2023-06-08T03:51:35.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 -- start start 2023-06-08T03:51:35.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9406f210 0x7fdd94152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd940c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9415f6c0 0x7fdd940c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9415d030 con 0x7fdd940cd6c0 2023-06-08T03:51:35.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940acee0 con 0x7fdd9415f6c0 2023-06-08T03:51:35.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd9a703700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940ad1c0 con 0x7fdd9406f210 2023-06-08T03:51:35.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd937fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd940c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd937fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd940c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37132/0 (socket says 172.21.15.79:37132) 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.790+0000 7fdd937fe700 1 -- 172.21.15.79:0/702072360 learned_addr learned my addr 172.21.15.79:0/702072360 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd937fe700 1 -- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9406f210 msgr2=0x7fdd94152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd937fe700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9406f210 0x7fdd94152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd937fe700 1 -- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9415f6c0 msgr2=0x7fdd940c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd937fe700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9415f6c0 0x7fdd940c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:35.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd937fe700 1 -- 172.21.15.79:0/702072360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840094a0 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd937fe700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd940c8cb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009b90 tx=0x7fdd84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840036a0 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8401f070 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd940c66e0 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84003a00 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.791+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd940c6ca0 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.792+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94066fb0 con 0x7fdd940cd6c0 2023-06-08T03:51:35.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.792+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd84003b60 con 0x7fdd940cd6c0 2023-06-08T03:51:35.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.792+0000 7fdd90ff9700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd7406ace0 0x7fdd7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:35.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.793+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdd84021070 con 0x7fdd940cd6c0 2023-06-08T03:51:35.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.793+0000 7fdd93fff700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd7406ace0 0x7fdd7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:35.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.793+0000 7fdd93fff700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd7406ace0 0x7fdd7406d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0098d0 tx=0x7fdd7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:35.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:35.797+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdd84055fe0 con 0x7fdd940cd6c0 2023-06-08T03:51:36.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.029+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd94073e90 con 0x7fdd940cd6c0 2023-06-08T03:51:36.033 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.030+0000 7fdd90ff9700 1 -- 172.21.15.79:0/702072360 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fdd84059600 con 0x7fdd940cd6c0 2023-06-08T03:51:36.034 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:36.034 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:51:36.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd7406ace0 msgr2=0x7fdd7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd7406ace0 0x7fdd7406d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0098d0 tx=0x7fdd7c008040 comp rx=0 tx=0).stop 2023-06-08T03:51:36.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 msgr2=0x7fdd940c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd940c8cb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009b90 tx=0x7fdd84004750 comp rx=0 tx=0).stop 2023-06-08T03:51:36.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 shutdown_connections 2023-06-08T03:51:36.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd9415f6c0 0x7fdd940c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fdd7406ace0 0x7fdd7406d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdd9406f210 0x7fdd94152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 --2- 172.21.15.79:0/702072360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd940cd6c0 0x7fdd940c8cb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.031+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 >> 172.21.15.79:0/702072360 conn(0x7fdd940b1d20 msgr2=0x7fdd9415ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.032+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 shutdown_connections 2023-06-08T03:51:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.032+0000 7fdd9a703700 1 -- 172.21.15.79:0/702072360 wait complete. 2023-06-08T03:51:36.037 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:51:36.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:36.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.163+0000 7f1daa969700 1 Processor -- start 2023-06-08T03:51:36.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.163+0000 7f1daa969700 1 -- start start 2023-06-08T03:51:36.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1daa969700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da40ca7c0 0x7f1da40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1daa969700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1da406b1c0 0x7f1da406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1daa969700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da40cf710 con 0x7f1da40707e0 2023-06-08T03:51:36.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1daa969700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da40cfc90 con 0x7f1da406b1c0 2023-06-08T03:51:36.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1daa969700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cf9b0 con 0x7f1da40ca7c0 2023-06-08T03:51:36.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1da406b1c0 0x7f1da406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1da406b1c0 0x7f1da406a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37140/0 (socket says 172.21.15.79:37140) 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 -- 172.21.15.79:0/3122709695 learned_addr learned my addr 172.21.15.79:0/3122709695 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da3fff700 1 --2- 172.21.15.79:0/3122709695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da40ca7c0 0x7f1da40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 -- 172.21.15.79:0/3122709695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da40ca7c0 msgr2=0x7f1da40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 --2- 172.21.15.79:0/3122709695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da40ca7c0 0x7f1da40cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 -- 172.21.15.79:0/3122709695 >> v1:172.21.15.156:6789/0 conn(0x7f1da40707e0 legacy=0x7f1da4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 -- 172.21.15.79:0/3122709695 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da4069ef0 con 0x7f1da406b1c0 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.164+0000 7f1da37fe700 1 --2- 172.21.15.79:0/3122709695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da406b1c0 0x7f1da406a6f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1d940098f0 tx=0x7f1d94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81264bc0edd7b729 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1da2ffd700 1 -- 172.21.15.79:0/3122709695 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004230 con 0x7f1da406b1c0 2023-06-08T03:51:36.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1da2ffd700 1 -- 172.21.15.79:0/3122709695 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94004390 con 0x7f1da406b1c0 2023-06-08T03:51:36.171 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1da2ffd700 1 -- 172.21.15.79:0/3122709695 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d940036a0 con 0x7f1da406b1c0 2023-06-08T03:51:36.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 -- 172.21.15.79:0/3122709695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da406b1c0 msgr2=0x7f1da406a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 --2- 172.21.15.79:0/3122709695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da406b1c0 0x7f1da406a6f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1d940098f0 tx=0x7f1d94009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:36.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 -- 172.21.15.79:0/3122709695 shutdown_connections 2023-06-08T03:51:36.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 --2- 172.21.15.79:0/3122709695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da40ca7c0 0x7f1da40cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 --2- 172.21.15.79:0/3122709695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da406b1c0 0x7f1da406a6f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 -- 172.21.15.79:0/3122709695 >> 172.21.15.79:0/3122709695 conn(0x7f1da40b14a0 msgr2=0x7f1da40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 -- 172.21.15.79:0/3122709695 shutdown_connections 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.165+0000 7f1daa969700 1 -- 172.21.15.79:0/3122709695 wait complete. 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.166+0000 7f1daa969700 1 Processor -- start 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 -- start start 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da406b1c0 0x7f1da41468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1da40707e0 0x7f1da40c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 0x7f1da4145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4144fe0 con 0x7f1da40ca7c0 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4158830 con 0x7f1da40707e0 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1daa969700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4146e30 con 0x7f1da406b1c0 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1da8f06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 0x7f1da4145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1da8f06700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 0x7f1da4145400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37148/0 (socket says 172.21.15.79:37148) 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.167+0000 7f1da8f06700 1 -- 172.21.15.79:0/4081014354 learned_addr learned my addr 172.21.15.79:0/4081014354 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da3fff700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da406b1c0 0x7f1da41468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da37fe700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1da40707e0 0x7f1da40c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da8f06700 1 -- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da406b1c0 msgr2=0x7f1da41468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da8f06700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da406b1c0 0x7f1da41468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da8f06700 1 -- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1da40707e0 msgr2=0x7f1da40c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da8f06700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1da40707e0 0x7f1da40c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da8f06700 1 -- 172.21.15.79:0/4081014354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94009550 con 0x7f1da40ca7c0 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da8f06700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 0x7f1da4145400 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00cc10 tx=0x7f1d9c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00ab10 con 0x7f1da40ca7c0 2023-06-08T03:51:36.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00ac70 con 0x7f1da40ca7c0 2023-06-08T03:51:36.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da4144360 con 0x7f1da40ca7c0 2023-06-08T03:51:36.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c012420 con 0x7f1da40ca7c0 2023-06-08T03:51:36.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.168+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da4144920 con 0x7f1da40ca7c0 2023-06-08T03:51:36.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.169+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da4145120 con 0x7f1da40ca7c0 2023-06-08T03:51:36.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.169+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d9c012580 con 0x7f1da40ca7c0 2023-06-08T03:51:36.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.170+0000 7f1da0ff9700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d9006ace0 0x7f1d9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.170+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1d9c089a50 con 0x7f1da40ca7c0 2023-06-08T03:51:36.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.170+0000 7f1da3fff700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d9006ace0 0x7f1d9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.171+0000 7f1da3fff700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d9006ace0 0x7f1d9006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007a30 tx=0x7f1d98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.179 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.174+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1da4145120 con 0x7f1da40ca7c0 2023-06-08T03:51:36.368 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.365+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1da4145120 con 0x7f1da40ca7c0 2023-06-08T03:51:36.369 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.366+0000 7f1da0ff9700 1 -- 172.21.15.79:0/4081014354 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f1da4145120 con 0x7f1da40ca7c0 2023-06-08T03:51:36.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:36.370 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:36.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d9006ace0 msgr2=0x7f1d9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.372 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d9006ace0 0x7f1d9006d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1d98007a30 tx=0x7f1d98008040 comp rx=0 tx=0).stop 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 msgr2=0x7f1da4145400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 0x7f1da4145400 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00cc10 tx=0x7f1d9c00cf20 comp rx=0 tx=0).stop 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 shutdown_connections 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1da40ca7c0 0x7f1da4145400 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1d9006ace0 0x7f1d9006d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1da406b1c0 0x7f1da41468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.373 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 --2- 172.21.15.79:0/4081014354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1da40707e0 0x7f1da40c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 >> 172.21.15.79:0/4081014354 conn(0x7f1da40b14a0 msgr2=0x7f1da406d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:36.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 shutdown_connections 2023-06-08T03:51:36.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.368+0000 7f1daa969700 1 -- 172.21.15.79:0/4081014354 wait complete. 2023-06-08T03:51:36.384 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-08T03:51:36.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:51:36.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.500+0000 7f6462559700 1 Processor -- start 2023-06-08T03:51:36.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.500+0000 7f6462559700 1 -- start start 2023-06-08T03:51:36.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f6462559700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f645c0cd090 0x7f645c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f6462559700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c0b0340 0x7f645c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f6462559700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0ce250 con 0x7f645c15d7a0 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f6462559700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0cda40 con 0x7f645c0cd090 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f6462559700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0afeb0 con 0x7f645c0b0340 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645b7fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f645c0cd090 0x7f645c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645bfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f645c15d7a0 0x7f645c0cc5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60628/0 (socket says 172.21.15.79:60628) 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645bfff700 1 -- 172.21.15.79:0/3536725034 learned_addr learned my addr 172.21.15.79:0/3536725034 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:36.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f6460af6700 1 --2- 172.21.15.79:0/3536725034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c0b0340 0x7f645c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645b7fe700 1 -- 172.21.15.79:0/3536725034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c0b0340 msgr2=0x7f645c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645b7fe700 1 --2- 172.21.15.79:0/3536725034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c0b0340 0x7f645c0b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645b7fe700 1 -- 172.21.15.79:0/3536725034 >> v1:172.21.15.79:6789/0 conn(0x7f645c15d7a0 legacy=0x7f645c0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645b7fe700 1 -- 172.21.15.79:0/3536725034 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0ce4a0 con 0x7f645c0cd090 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.501+0000 7f645affd700 1 -- 172.21.15.79:0/3536725034 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39351307 0 0) 0x7f645c0ce250 con 0x7f645c15d7a0 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f645b7fe700 1 --2- 172.21.15.79:0/3536725034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0cd500 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f644c009840 tx=0x7f644c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53926707130bb958 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f645affd700 1 -- 172.21.15.79:0/3536725034 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c011830 con 0x7f645c0cd090 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f645affd700 1 -- 172.21.15.79:0/3536725034 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c011990 con 0x7f645c0cd090 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f645affd700 1 -- 172.21.15.79:0/3536725034 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c011d20 con 0x7f645c0cd090 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 -- 172.21.15.79:0/3536725034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 msgr2=0x7f645c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 --2- 172.21.15.79:0/3536725034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0cd500 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f644c009840 tx=0x7f644c009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 -- 172.21.15.79:0/3536725034 shutdown_connections 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 --2- 172.21.15.79:0/3536725034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c0b0340 0x7f645c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 --2- 172.21.15.79:0/3536725034 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0cd500 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 -- 172.21.15.79:0/3536725034 >> 172.21.15.79:0/3536725034 conn(0x7f645c099960 msgr2=0x7f645c0d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:36.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 -- 172.21.15.79:0/3536725034 shutdown_connections 2023-06-08T03:51:36.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.502+0000 7f6462559700 1 -- 172.21.15.79:0/3536725034 wait complete. 2023-06-08T03:51:36.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.503+0000 7f6462559700 1 Processor -- start 2023-06-08T03:51:36.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 -- start start 2023-06-08T03:51:36.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f645c0b0340 0x7f645c15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.512 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c15d7a0 0x7f645c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0c4610 con 0x7f645c0cd090 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0b49b0 con 0x7f645c0b0340 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.504+0000 7f6462559700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0b43f0 con 0x7f645c15d7a0 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37158/0 (socket says 172.21.15.79:37158) 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 -- 172.21.15.79:0/1114868935 learned_addr learned my addr 172.21.15.79:0/1114868935 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 -- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c15d7a0 msgr2=0x7f645c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.513 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c15d7a0 0x7f645c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 -- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f645c0b0340 msgr2=0x7f645c15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645bfff700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f645c0b0340 0x7f645c15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f645c0b0340 0x7f645c15a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 -- 172.21.15.79:0/1114868935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0094a0 con 0x7f645c0cd090 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f645b7fe700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0c6ae0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0045f0 con 0x7f645c0cd090 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c01f070 con 0x7f645c0cd090 2023-06-08T03:51:36.514 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0b46d0 con 0x7f645c0cd090 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c003ae0 con 0x7f645c0cd090 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.505+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c15aae0 con 0x7f645c0cd090 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.506+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c0c4750 con 0x7f645c0cd090 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.507+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f644c003c40 con 0x7f645c0cd090 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.507+0000 7f6458ff9700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f644806ace0 0x7f644806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.507+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f644c021070 con 0x7f645c0cd090 2023-06-08T03:51:36.515 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.507+0000 7f645bfff700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f644806ace0 0x7f644806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.508+0000 7f645bfff700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f644806ace0 0x7f644806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6450000f60 tx=0x7f6450009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.516 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.512+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f645c0c4750 con 0x7f645c0cd090 2023-06-08T03:51:36.746 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.742+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f645c0c4750 con 0x7f645c0cd090 2023-06-08T03:51:36.747 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.742+0000 7f6458ff9700 1 -- 172.21.15.79:0/1114868935 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f645c0c4750 con 0x7f645c0cd090 2023-06-08T03:51:36.747 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:36.747 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:51:36.748 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f644806ace0 msgr2=0x7f644806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f644806ace0 0x7f644806d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6450000f60 tx=0x7f6450009040 comp rx=0 tx=0).stop 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 msgr2=0x7f645c0c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0c6ae0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 shutdown_connections 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f644806ace0 0x7f644806d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f645c15d7a0 0x7f645c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.749 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f645c0b0340 0x7f645c15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 --2- 172.21.15.79:0/1114868935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f645c0cd090 0x7f645c0c6ae0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.744+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 >> 172.21.15.79:0/1114868935 conn(0x7f645c099960 msgr2=0x7f645c0af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:36.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.745+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 shutdown_connections 2023-06-08T03:51:36.750 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.745+0000 7f6462559700 1 -- 172.21.15.79:0/1114868935 wait complete. 2023-06-08T03:51:36.750 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:51:36.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:51:36.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 Processor -- start 2023-06-08T03:51:36.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 -- start start 2023-06-08T03:51:36.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe36c15a640 0x7fe36c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe36c15aa10 0x7fe36c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0b0750 con 0x7fe36c15a640 2023-06-08T03:51:36.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c0ceeb0 con 0x7fe36c15aa10 2023-06-08T03:51:36.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe37264c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0b09a0 con 0x7fe36c0ce930 2023-06-08T03:51:36.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.877+0000 7fe36bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe36c15a640 0x7fe36c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe36c15a640 0x7fe36c0b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37166/0 (socket says 172.21.15.79:37166) 2023-06-08T03:51:36.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 -- 172.21.15.79:0/541881509 learned_addr learned my addr 172.21.15.79:0/541881509 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:36.883 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe370be9700 1 --2- 172.21.15.79:0/541881509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/541881509 >> v2:172.21.15.156:3300/0 conn(0x7fe36c15aa10 0x7fe36c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 -- 172.21.15.79:0/541881509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 msgr2=0x7fe36c0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 --2- 172.21.15.79:0/541881509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c0af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 -- 172.21.15.79:0/541881509 >> v2:172.21.15.156:3300/0 conn(0x7fe36c15aa10 msgr2=0x7fe36c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 --2- 172.21.15.79:0/541881509 >> v2:172.21.15.156:3300/0 conn(0x7fe36c15aa10 0x7fe36c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 -- 172.21.15.79:0/541881509 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0b0c80 con 0x7fe36c15a640 2023-06-08T03:51:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/541881509 >> v2:172.21.15.156:3300/0 conn(0x7fe36c15aa10 0x7fe36c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36bfff700 1 --2- 172.21.15.79:0/541881509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c0b1480 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe36c071af0 tx=0x7fe360009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde6a5866b627c5c server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36affd700 1 -- 172.21.15.79:0/541881509 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360011830 con 0x7fe36c15a640 2023-06-08T03:51:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36affd700 1 -- 172.21.15.79:0/541881509 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360011990 con 0x7fe36c15a640 2023-06-08T03:51:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.878+0000 7fe36affd700 1 -- 172.21.15.79:0/541881509 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360011d20 con 0x7fe36c15a640 2023-06-08T03:51:36.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 -- 172.21.15.79:0/541881509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 msgr2=0x7fe36c0b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 --2- 172.21.15.79:0/541881509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c0b1480 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe36c071af0 tx=0x7fe360009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 -- 172.21.15.79:0/541881509 shutdown_connections 2023-06-08T03:51:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 --2- 172.21.15.79:0/541881509 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 --2- 172.21.15.79:0/541881509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c0b1480 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 --2- 172.21.15.79:0/541881509 >> v2:172.21.15.156:3300/0 conn(0x7fe36c15aa10 0x7fe36c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 -- 172.21.15.79:0/541881509 >> 172.21.15.79:0/541881509 conn(0x7fe36c06a670 msgr2=0x7fe36c099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 -- 172.21.15.79:0/541881509 shutdown_connections 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.879+0000 7fe37264c700 1 -- 172.21.15.79:0/541881509 wait complete. 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.880+0000 7fe37264c700 1 Processor -- start 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.880+0000 7fe37264c700 1 -- start start 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c06f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c15aa10 0x7fe36c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1f1e20 con 0x7fe36c15a640 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1f2c40 con 0x7fe36c15aa10 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1f2030 con 0x7fe36c0ce930 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c06f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c06f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37174/0 (socket says 172.21.15.79:37174) 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 -- 172.21.15.79:0/4060529353 learned_addr learned my addr 172.21.15.79:0/4060529353 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe370be9700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c15aa10 0x7fe36c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36bfff700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 -- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 msgr2=0x7fe36c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 -- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c15aa10 msgr2=0x7fe36c06fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c15aa10 0x7fe36c06fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 -- 172.21.15.79:0/4060529353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360009550 con 0x7fe36c15a640 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36bfff700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe36b7fe700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c06f630 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00e810 tx=0x7fe35c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00c020 con 0x7fe36c15a640 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c004250 con 0x7fe36c15a640 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c1f2310 con 0x7fe36c15a640 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0036a0 con 0x7fe36c15a640 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.881+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c1f28d0 con 0x7fe36c15a640 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.882+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c1f2df0 con 0x7fe36c15a640 2023-06-08T03:51:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.883+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe35c003800 con 0x7fe36c15a640 2023-06-08T03:51:36.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.883+0000 7fe368ff9700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe35806ace0 0x7fe35806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:36.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.883+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe35c089490 con 0x7fe36c15a640 2023-06-08T03:51:36.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.884+0000 7fe36bfff700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe35806ace0 0x7fe35806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:36.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.884+0000 7fe36bfff700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe35806ace0 0x7fe35806d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe360009fd0 tx=0x7fe360008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:36.893 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:36.888+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe36c1f2df0 con 0x7fe36c15a640 2023-06-08T03:51:37.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.120+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe36c1f2db0 con 0x7fe36c15a640 2023-06-08T03:51:37.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.120+0000 7fe368ff9700 1 -- 172.21.15.79:0/4060529353 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fe35c057d70 con 0x7fe36c15a640 2023-06-08T03:51:37.124 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:37.125 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/1755519366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1755519366},{"type":"v1","addr":"172.21.15.156:6836","nonce":1755519366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4246,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/769535759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":769535759},{"type":"v1","addr":"172.21.15.79:6835","nonce":769535759}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2988640239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2988640239},{"type":"v1","addr":"172.21.15.156:6837","nonce":2988640239}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:50:24.668853+0000","modified":"2023-06-08T03:50:41.945158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6833/2931327716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":2931327716},{"type":"v1","addr":"172.21.15.79:6833","nonce":2931327716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-08T03:51:37.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe35806ace0 msgr2=0x7fe35806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe35806ace0 0x7fe35806d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe360009fd0 tx=0x7fe360008030 comp rx=0 tx=0).stop 2023-06-08T03:51:37.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 msgr2=0x7fe36c06f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c06f630 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00e810 tx=0x7fe35c00ebd0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 shutdown_connections 2023-06-08T03:51:37.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe36c15aa10 0x7fe36c06fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fe35806ace0 0x7fe35806d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe36c0ce930 0x7fe36c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 --2- 172.21.15.79:0/4060529353 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe36c15a640 0x7fe36c06f630 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 >> 172.21.15.79:0/4060529353 conn(0x7fe36c06a670 msgr2=0x7fe36c0cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 shutdown_connections 2023-06-08T03:51:37.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.123+0000 7fe37264c700 1 -- 172.21.15.79:0/4060529353 wait complete. 2023-06-08T03:51:37.129 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2023-06-08T03:51:37.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:37.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.254+0000 7f515daf5700 1 Processor -- start 2023-06-08T03:51:37.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f515daf5700 1 -- start start 2023-06-08T03:51:37.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f515daf5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5158071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f515daf5700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51580cee20 con 0x7f515806a460 2023-06-08T03:51:37.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f515daf5700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515806fa00 con 0x7f515815f6c0 2023-06-08T03:51:37.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f515daf5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580cf330 con 0x7f51580cd6c0 2023-06-08T03:51:37.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f5156ffd700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f515806a460 0x7f5158071050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60656/0 (socket says 172.21.15.79:60656) 2023-06-08T03:51:37.262 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.255+0000 7f5156ffd700 1 -- 172.21.15.79:0/2902560661 learned_addr learned my addr 172.21.15.79:0/2902560661 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f5157fff700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5158071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511474783 0 0) 0x7f51580cee20 con 0x7f515806a460 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51440035b0 con 0x7f515806a460 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3158002871 0 0) 0x7f51440035b0 con 0x7f515806a460 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 msgr2=0x7f5158071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5158071760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 >> v1:172.21.15.156:6789/0 conn(0x7f515815f6c0 legacy=0x7f5158069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.263 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580cd1e0 con 0x7f515806a460 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f5140001670 con 0x7f515806a460 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 >> v1:172.21.15.79:6789/0 conn(0x7f515806a460 legacy=0x7f5158071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144003aa0 0x7f5144003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144004850 0x7f5144004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5144005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51580cf330 con 0x7f5144003aa0 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514400baf0 con 0x7f5144004850 2023-06-08T03:51:37.264 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514400bcb0 con 0x7f51580cd6c0 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51577fe700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144003aa0 0x7f5144003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f5156ffd700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144004850 0x7f5144004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f5157fff700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5144005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51577fe700 1 -- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 msgr2=0x7f5144005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51577fe700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5144005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51577fe700 1 -- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144004850 msgr2=0x7f5144004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:37.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51577fe700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5144004850 0x7f5144004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f51577fe700 1 -- 172.21.15.79:0/2902560661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440035b0 con 0x7f5144003aa0 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.257+0000 7f5157fff700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51580cd6c0 0x7f5144005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f51577fe700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144003aa0 0x7f5144003eb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f51480017c0 tx=0x7f5144005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440b6dde8487ea36 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480052d0 con 0x7f5144003aa0 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5148005430 con 0x7f5144003aa0 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f51567fc700 1 -- 172.21.15.79:0/2902560661 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480038c0 con 0x7f5144003aa0 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f515daf5700 1 -- 172.21.15.79:0/2902560661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144003aa0 msgr2=0x7f5144003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f515daf5700 1 --2- 172.21.15.79:0/2902560661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5144003aa0 0x7f5144003eb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f51480017c0 tx=0x7f5144005880 comp rx=0 tx=0).stop 2023-06-08T03:51:37.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f5157fff700 1 -- 172.21.15.79:0/2902560661 reap_dead start 2023-06-08T03:51:37.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f515daf5700 1 -- 172.21.15.79:0/2902560661 shutdown_connections 2023-06-08T03:51:37.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f515daf5700 1 -- 172.21.15.79:0/2902560661 >> 172.21.15.79:0/2902560661 conn(0x7f51580b1d20 msgr2=0x7f515806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:37.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f515daf5700 1 -- 172.21.15.79:0/2902560661 shutdown_connections 2023-06-08T03:51:37.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.258+0000 7f515daf5700 1 -- 172.21.15.79:0/2902560661 wait complete. 2023-06-08T03:51:37.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 Processor -- start 2023-06-08T03:51:37.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 -- start start 2023-06-08T03:51:37.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5144003aa0 0x7f5158152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51580b3e10 0x7f51580af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 0x7f51580afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51580ae530 con 0x7f5158153ad0 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51580adc00 con 0x7f51580b3e10 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f515daf5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580adee0 con 0x7f5144003aa0 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f5157fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 0x7f51580afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f5157fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 0x7f51580afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37196/0 (socket says 172.21.15.79:37196) 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.259+0000 7f5157fff700 1 -- 172.21.15.79:0/317466174 learned_addr learned my addr 172.21.15.79:0/317466174 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f51577fe700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5144003aa0 0x7f5158152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5156ffd700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51580b3e10 0x7f51580af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5157fff700 1 -- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5144003aa0 msgr2=0x7f5158152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5157fff700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5144003aa0 0x7f5158152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5157fff700 1 -- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51580b3e10 msgr2=0x7f51580af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5157fff700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51580b3e10 0x7f51580af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5157fff700 1 -- 172.21.15.79:0/317466174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580ae1c0 con 0x7f5158153ad0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f5157fff700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 0x7f51580afa60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f515806ae10 tx=0x7f51500104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515000e050 con 0x7f5158153ad0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515000f040 con 0x7f5158153ad0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150010e10 con 0x7f5158153ad0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51580ad8b0 con 0x7f5158153ad0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.260+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51581536d0 con 0x7f5158153ad0 2023-06-08T03:51:37.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.262+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5158073e90 con 0x7f5158153ad0 2023-06-08T03:51:37.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.262+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5150015440 con 0x7f5158153ad0 2023-06-08T03:51:37.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.263+0000 7f514ffff700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f513806ace0 0x7f513806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.263+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5150089870 con 0x7f5158153ad0 2023-06-08T03:51:37.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.263+0000 7f51577fe700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f513806ace0 0x7f513806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.263+0000 7f51577fe700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f513806ace0 0x7f513806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5148000c00 tx=0x7f5148009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:37.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.267+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51500546c0 con 0x7f5158153ad0 2023-06-08T03:51:37.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.453+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51580ae350 con 0x7f5158153ad0 2023-06-08T03:51:37.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.454+0000 7f514ffff700 1 -- 172.21.15.79:0/317466174 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+20110 (secure 0 0 0) 0x7f5150057ce0 con 0x7f5158153ad0 2023-06-08T03:51:37.458 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:37.458 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:50:29.304160+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:50:22.311526+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-08T03:50:23.630403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-06-08T03:50:26.911508+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:37.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f513806ace0 msgr2=0x7f513806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f513806ace0 0x7f513806d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5148000c00 tx=0x7f5148009030 comp rx=0 tx=0).stop 2023-06-08T03:51:37.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 msgr2=0x7f51580afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 0x7f51580afa60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f515806ae10 tx=0x7f51500104a0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 shutdown_connections 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5158153ad0 0x7f51580afa60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f513806ace0 0x7f513806d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5144003aa0 0x7f5158152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 --2- 172.21.15.79:0/317466174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51580b3e10 0x7f51580af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 >> 172.21.15.79:0/317466174 conn(0x7f51580b1d20 msgr2=0x7f515806eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 shutdown_connections 2023-06-08T03:51:37.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.457+0000 7f515daf5700 1 -- 172.21.15.79:0/317466174 wait complete. 2023-06-08T03:51:37.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-08T03:51:37.592 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.589+0000 7f8a32b1e700 1 Processor -- start 2023-06-08T03:51:37.593 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a32b1e700 1 -- start start 2023-06-08T03:51:37.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a32b1e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c159100 0x7f8a2c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a32b1e700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8a2c06d6a0 0x7f8a2c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a32b1e700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c0cc5c0 con 0x7f8a2c158d30 2023-06-08T03:51:37.594 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a32b1e700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2c06dac0 con 0x7f8a2c06d6a0 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a32b1e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c15d470 con 0x7f8a2c159100 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a308ba700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8a2c158d30 0x7f8a2c15c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60676/0 (socket says 172.21.15.79:60676) 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a308ba700 1 -- 172.21.15.79:0/1226378769 learned_addr learned my addr 172.21.15.79:0/1226378769 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.590+0000 7f8a310bb700 1 --2- 172.21.15.79:0/1226378769 >> v2:172.21.15.79:3300/0 conn(0x7f8a2c06d6a0 0x7f8a2c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a2bfff700 1 --2- 172.21.15.79:0/1226378769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c159100 0x7f8a2c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a2b7fe700 1 -- 172.21.15.79:0/1226378769 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790823820 0 0) 0x7f8a2c0cc5c0 con 0x7f8a2c158d30 2023-06-08T03:51:37.595 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a2b7fe700 1 -- 172.21.15.79:0/1226378769 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a1c0035b0 con 0x7f8a2c158d30 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a310bb700 1 -- 172.21.15.79:0/1226378769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c159100 msgr2=0x7f8a2c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a310bb700 1 --2- 172.21.15.79:0/1226378769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c159100 0x7f8a2c06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a310bb700 1 -- 172.21.15.79:0/1226378769 >> v1:172.21.15.79:6789/0 conn(0x7f8a2c158d30 legacy=0x7f8a2c15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a310bb700 1 -- 172.21.15.79:0/1226378769 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c15d750 con 0x7f8a2c06d6a0 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a310bb700 1 --2- 172.21.15.79:0/1226378769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c06d6a0 0x7f8a2c0cd010 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009d90 tx=0x7f8a2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bdc94790dd1454 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a2b7fe700 1 -- 172.21.15.79:0/1226378769 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2400e070 con 0x7f8a2c06d6a0 2023-06-08T03:51:37.596 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a2b7fe700 1 -- 172.21.15.79:0/1226378769 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2400bb20 con 0x7f8a2c06d6a0 2023-06-08T03:51:37.597 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.591+0000 7f8a2b7fe700 1 -- 172.21.15.79:0/1226378769 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a24012430 con 0x7f8a2c06d6a0 2023-06-08T03:51:37.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 -- 172.21.15.79:0/1226378769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c06d6a0 msgr2=0x7f8a2c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/1226378769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c06d6a0 0x7f8a2c0cd010 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009d90 tx=0x7f8a2400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:37.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 -- 172.21.15.79:0/1226378769 shutdown_connections 2023-06-08T03:51:37.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/1226378769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c06d6a0 0x7f8a2c0cd010 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.599 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/1226378769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c159100 0x7f8a2c06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 -- 172.21.15.79:0/1226378769 >> 172.21.15.79:0/1226378769 conn(0x7f8a2c0ce4e0 msgr2=0x7f8a2c0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 -- 172.21.15.79:0/1226378769 shutdown_connections 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.592+0000 7f8a32b1e700 1 -- 172.21.15.79:0/1226378769 wait complete. 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.593+0000 7f8a32b1e700 1 Processor -- start 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.593+0000 7f8a32b1e700 1 -- start start 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a32b1e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c06d6a0 0x7f8a2c1f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.600 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a32b1e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c158d30 0x7f8a2c1f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a32b1e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 0x7f8a2c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a32b1e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1f0960 con 0x7f8a2c159100 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a32b1e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1f3ec0 con 0x7f8a2c158d30 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a32b1e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c0cdf20 con 0x7f8a2c06d6a0 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 0x7f8a2c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 0x7f8a2c0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37208/0 (socket says 172.21.15.79:37208) 2023-06-08T03:51:37.601 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 -- 172.21.15.79:0/702957310 learned_addr learned my addr 172.21.15.79:0/702957310 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a308ba700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c06d6a0 0x7f8a2c1f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a2bfff700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c158d30 0x7f8a2c1f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 -- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c06d6a0 msgr2=0x7f8a2c1f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c06d6a0 0x7f8a2c1f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 -- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c158d30 msgr2=0x7f8a2c1f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c158d30 0x7f8a2c1f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 -- 172.21.15.79:0/702957310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24009a70 con 0x7f8a2c159100 2023-06-08T03:51:37.602 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.594+0000 7f8a310bb700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 0x7f8a2c0b3d80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8a24014040 tx=0x7f8a24004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.595+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240129f0 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.595+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a24020070 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.595+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a24012db0 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.595+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c0b1e70 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.595+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c0b1b90 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.595+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c0c98e0 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.596+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a24004040 con 0x7f8a2c159100 2023-06-08T03:51:37.603 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.596+0000 7f8a297fa700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8a1006ace0 0x7f8a1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:37.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.596+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a24022070 con 0x7f8a2c159100 2023-06-08T03:51:37.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.596+0000 7f8a308ba700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8a1006ace0 0x7f8a1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:37.604 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.597+0000 7f8a308ba700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8a1006ace0 0x7f8a1006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a20007fd0 tx=0x7f8a20007dc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:37.605 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.601+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8a24056130 con 0x7f8a2c159100 2023-06-08T03:51:37.835 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:37.832+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8a2c073e90 con 0x7f8a2c159100 2023-06-08T03:51:38.189 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-06-08T03:51:38.187+0000 7f9fb6f8b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-08T03:51:38.193 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.190+0000 7f8a297fa700 1 -- 172.21.15.79:0/702957310 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8a24059750 con 0x7f8a2c159100 2023-06-08T03:51:38.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8a1006ace0 msgr2=0x7f8a1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:38.195 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8a1006ace0 0x7f8a1006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a20007fd0 tx=0x7f8a20007dc0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 msgr2=0x7f8a2c0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 0x7f8a2c0b3d80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8a24014040 tx=0x7f8a24004ad0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 shutdown_connections 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8a2c159100 0x7f8a2c0b3d80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8a1006ace0 0x7f8a1006d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8a2c06d6a0 0x7f8a2c1f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.196 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 --2- 172.21.15.79:0/702957310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a2c158d30 0x7f8a2c1f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 >> 172.21.15.79:0/702957310 conn(0x7f8a2c0ce4e0 msgr2=0x7f8a2c0cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:38.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 shutdown_connections 2023-06-08T03:51:38.197 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.192+0000 7f8a32b1e700 1 -- 172.21.15.79:0/702957310 wait complete. 2023-06-08T03:51:38.197 INFO:teuthology.orchestra.run.smithi079.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-08T03:51:38.197 INFO:tasks.ceph.mds.a.smithi079.stderr: -5177> 2023-06-08T03:50:30.315+0000 7f12daea6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-08T03:51:38.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-08T03:51:38.227 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2023-06-08T03:51:38.334 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 Processor -- start 2023-06-08T03:51:38.335 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 -- start start 2023-06-08T03:51:38.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9d4815c1f0 0x7f9d4806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d48071fe0 0x7f9d480cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9d480cf130 0x7f9d48069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480cd140 con 0x7f9d4815c1f0 2023-06-08T03:51:38.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d48071110 con 0x7f9d480cf130 2023-06-08T03:51:38.337 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d4e814700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48071be0 con 0x7f9d48071fe0 2023-06-08T03:51:38.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.332+0000 7f9d47fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9d4815c1f0 0x7f9d4806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9d4815c1f0 0x7f9d4806d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37226/0 (socket says 172.21.15.79:37226) 2023-06-08T03:51:38.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 -- 172.21.15.79:0/3332739060 learned_addr learned my addr 172.21.15.79:0/3332739060 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:38.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d4cdb1700 1 --2- 172.21.15.79:0/3332739060 >> v2:172.21.15.156:3300/0 conn(0x7f9d480cf130 0x7f9d48069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d477fe700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d48071fe0 0x7f9d480cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.338 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 -- 172.21.15.79:0/3332739060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d48071fe0 msgr2=0x7f9d480cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d48071fe0 0x7f9d480cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 -- 172.21.15.79:0/3332739060 >> v2:172.21.15.156:3300/0 conn(0x7f9d480cf130 msgr2=0x7f9d48069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 --2- 172.21.15.79:0/3332739060 >> v2:172.21.15.156:3300/0 conn(0x7f9d480cf130 0x7f9d48069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 -- 172.21.15.79:0/3332739060 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480cd390 con 0x7f9d4815c1f0 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d477fe700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d48071fe0 0x7f9d480cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d47fff700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4815c1f0 0x7f9d4806d8a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d48070e40 tx=0x7f9d3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6e52a2222fd7ec server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d46ffd700 1 -- 172.21.15.79:0/3332739060 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00b870 con 0x7f9d4815c1f0 2023-06-08T03:51:38.339 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d46ffd700 1 -- 172.21.15.79:0/3332739060 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c00b9d0 con 0x7f9d4815c1f0 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.333+0000 7f9d46ffd700 1 -- 172.21.15.79:0/3332739060 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c00bd60 con 0x7f9d4815c1f0 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 -- 172.21.15.79:0/3332739060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4815c1f0 msgr2=0x7f9d4806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4815c1f0 0x7f9d4806d8a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9d48070e40 tx=0x7f9d3c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 -- 172.21.15.79:0/3332739060 shutdown_connections 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 --2- 172.21.15.79:0/3332739060 >> v2:172.21.15.156:3300/0 conn(0x7f9d480cf130 0x7f9d48069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d4815c1f0 0x7f9d4806d8a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.340 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 --2- 172.21.15.79:0/3332739060 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d48071fe0 0x7f9d480cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 -- 172.21.15.79:0/3332739060 >> 172.21.15.79:0/3332739060 conn(0x7f9d480b1d30 msgr2=0x7f9d4806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:38.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 -- 172.21.15.79:0/3332739060 shutdown_connections 2023-06-08T03:51:38.341 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.334+0000 7f9d4e814700 1 -- 172.21.15.79:0/3332739060 wait complete. 2023-06-08T03:51:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.335+0000 7f9d4e814700 1 Processor -- start 2023-06-08T03:51:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.335+0000 7f9d4e814700 1 -- start start 2023-06-08T03:51:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4e814700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 0x7f9d480c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4e814700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d480cf130 0x7f9d480b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.343 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4e814700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d4815c1f0 0x7f9d481405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4e814700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480c6e50 con 0x7f9d48071fe0 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4e814700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480c3c80 con 0x7f9d4815c1f0 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4e814700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d481f1b10 con 0x7f9d480cf130 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 0x7f9d480c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 0x7f9d480c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37232/0 (socket says 172.21.15.79:37232) 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 -- 172.21.15.79:0/4057009662 learned_addr learned my addr 172.21.15.79:0/4057009662 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:38.344 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d477fe700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d480cf130 0x7f9d480b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d4cdb1700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d4815c1f0 0x7f9d481405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 -- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d480cf130 msgr2=0x7f9d480b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d480cf130 0x7f9d480b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 -- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d4815c1f0 msgr2=0x7f9d481405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d4815c1f0 0x7f9d481405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 -- 172.21.15.79:0/4057009662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c009550 con 0x7f9d48071fe0 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d477fe700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d480cf130 0x7f9d480b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:38.345 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d47fff700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 0x7f9d480c4690 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c000c00 tx=0x7f9d3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c004520 con 0x7f9d48071fe0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.336+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3c01e070 con 0x7f9d48071fe0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.337+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480c6640 con 0x7f9d48071fe0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.337+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0038c0 con 0x7f9d48071fe0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.337+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d480c6b70 con 0x7f9d48071fe0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.337+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d480c44e0 con 0x7f9d48071fe0 2023-06-08T03:51:38.346 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.338+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d3c003a20 con 0x7f9d48071fe0 2023-06-08T03:51:38.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.338+0000 7f9d44ff9700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9d3406ace0 0x7f9d3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:38.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.338+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f9d3c020070 con 0x7f9d48071fe0 2023-06-08T03:51:38.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.338+0000 7f9d477fe700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9d3406ace0 0x7f9d3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:38.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.339+0000 7f9d477fe700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9d3406ace0 0x7f9d3406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d481f5120 tx=0x7f9d38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:38.347 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.342+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d3c055a10 con 0x7f9d48071fe0 2023-06-08T03:51:38.575 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:38.573+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9d48073e90 con 0x7f9d48071fe0 2023-06-08T03:51:39.200 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.197+0000 7f9d44ff9700 1 -- 172.21.15.79:0/4057009662 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9d3c059030 con 0x7f9d48071fe0 2023-06-08T03:51:39.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9d3406ace0 msgr2=0x7f9d3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:39.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9d3406ace0 0x7f9d3406d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d481f5120 tx=0x7f9d38008040 comp rx=0 tx=0).stop 2023-06-08T03:51:39.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 msgr2=0x7f9d480c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:39.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 0x7f9d480c4690 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c000c00 tx=0x7f9d3c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:39.203 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 shutdown_connections 2023-06-08T03:51:39.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d4815c1f0 0x7f9d481405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9d48071fe0 0x7f9d480c4690 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9d3406ace0 0x7f9d3406d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 --2- 172.21.15.79:0/4057009662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9d480cf130 0x7f9d480b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.204 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 >> 172.21.15.79:0/4057009662 conn(0x7f9d480b1d30 msgr2=0x7f9d4815e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:39.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 shutdown_connections 2023-06-08T03:51:39.205 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.200+0000 7f9d4e814700 1 -- 172.21.15.79:0/4057009662 wait complete. 2023-06-08T03:51:39.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-08T03:51:39.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.381+0000 7f4127ac7700 1 Processor -- start 2023-06-08T03:51:39.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.381+0000 7f4127ac7700 1 -- start start 2023-06-08T03:51:39.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4127ac7700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41200cb620 0x7f412006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:39.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4127ac7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200b16d0 0x7f4120159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4127ac7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412006bb00 con 0x7f41200cefb0 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4127ac7700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412015d540 con 0x7f41200cb620 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4127ac7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412006ebe0 con 0x7f41200b16d0 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125863700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f41200cb620 0x7f412006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125062700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f41200cefb0 0x7f41200b0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60718/0 (socket says 172.21.15.79:60718) 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125062700 1 -- 172.21.15.79:0/599365269 learned_addr learned my addr 172.21.15.79:0/599365269 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:39.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4126064700 1 --2- 172.21.15.79:0/599365269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200b16d0 0x7f4120159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125863700 1 -- 172.21.15.79:0/599365269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200b16d0 msgr2=0x7f4120159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125863700 1 --2- 172.21.15.79:0/599365269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200b16d0 0x7f4120159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125863700 1 -- 172.21.15.79:0/599365269 >> v1:172.21.15.79:6789/0 conn(0x7f41200cefb0 legacy=0x7f41200b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125863700 1 -- 172.21.15.79:0/599365269 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412006eec0 con 0x7f41200cb620 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4124861700 1 -- 172.21.15.79:0/599365269 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079542271 0 0) 0x7f412006bb00 con 0x7f41200cefb0 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4125863700 1 --2- 172.21.15.79:0/599365269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200cb620 0x7f412006b220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f411c009840 tx=0x7f411c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6518acf6d9d16af server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:39.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4124861700 1 -- 172.21.15.79:0/599365269 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c011830 con 0x7f41200cb620 2023-06-08T03:51:39.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4124861700 1 -- 172.21.15.79:0/599365269 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c011990 con 0x7f41200cb620 2023-06-08T03:51:39.388 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.382+0000 7f4124861700 1 -- 172.21.15.79:0/599365269 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c011d20 con 0x7f41200cb620 2023-06-08T03:51:39.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 -- 172.21.15.79:0/599365269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200cb620 msgr2=0x7f412006b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:39.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 --2- 172.21.15.79:0/599365269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200cb620 0x7f412006b220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f411c009840 tx=0x7f411c009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 -- 172.21.15.79:0/599365269 shutdown_connections 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 --2- 172.21.15.79:0/599365269 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200b16d0 0x7f4120159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 --2- 172.21.15.79:0/599365269 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200cb620 0x7f412006b220 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 -- 172.21.15.79:0/599365269 >> 172.21.15.79:0/599365269 conn(0x7f41200cbfc0 msgr2=0x7f41200cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 -- 172.21.15.79:0/599365269 shutdown_connections 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.383+0000 7f4127ac7700 1 -- 172.21.15.79:0/599365269 wait complete. 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.384+0000 7f4127ac7700 1 Processor -- start 2023-06-08T03:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.384+0000 7f4127ac7700 1 -- start start 2023-06-08T03:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4127ac7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 0x7f41200ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4127ac7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41200cb620 0x7f41200c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4127ac7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200cefb0 0x7f412014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4127ac7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200acaf0 con 0x7f41200b16d0 2023-06-08T03:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4127ac7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200abee0 con 0x7f41200cb620 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4127ac7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41200ac1c0 con 0x7f41200cefb0 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 0x7f41200ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 0x7f41200ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37250/0 (socket says 172.21.15.79:37250) 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 -- 172.21.15.79:0/1420830647 learned_addr learned my addr 172.21.15.79:0/1420830647 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 -- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200cefb0 msgr2=0x7f412014d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200cefb0 0x7f412014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125062700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41200cb620 0x7f41200c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 -- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41200cb620 msgr2=0x7f41200c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41200cb620 0x7f41200c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.385+0000 7f4125863700 1 -- 172.21.15.79:0/1420830647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0094a0 con 0x7f41200b16d0 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.386+0000 7f4125863700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 0x7f41200ab8c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f411c009b90 tx=0x7f411c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.386+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0036a0 con 0x7f41200b16d0 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.386+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c01f070 con 0x7f41200b16d0 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.386+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c003a40 con 0x7f41200b16d0 2023-06-08T03:51:39.394 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.386+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41200ac4a0 con 0x7f41200b16d0 2023-06-08T03:51:39.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.386+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41200c56a0 con 0x7f41200b16d0 2023-06-08T03:51:39.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.387+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120073e90 con 0x7f41200b16d0 2023-06-08T03:51:39.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.387+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f411c003ba0 con 0x7f41200b16d0 2023-06-08T03:51:39.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.387+0000 7f41167fc700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f410406ace0 0x7f410406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:39.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.388+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f411c021070 con 0x7f41200b16d0 2023-06-08T03:51:39.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.388+0000 7f4125062700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f410406ace0 0x7f410406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.388+0000 7f4125062700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f410406ace0 0x7f410406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41200b0cc0 tx=0x7f4110009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.392+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f411c055f30 con 0x7f41200b16d0 2023-06-08T03:51:39.581 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:39.578+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4120071900 con 0x7f41200b16d0 2023-06-08T03:51:40.207 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.203+0000 7f41167fc700 1 -- 172.21.15.79:0/1420830647 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f411c059550 con 0x7f41200b16d0 2023-06-08T03:51:40.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f410406ace0 msgr2=0x7f410406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:40.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f410406ace0 0x7f410406d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41200b0cc0 tx=0x7f4110009040 comp rx=0 tx=0).stop 2023-06-08T03:51:40.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 msgr2=0x7f41200ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:40.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 0x7f41200ab8c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f411c009b90 tx=0x7f411c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:40.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 shutdown_connections 2023-06-08T03:51:40.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f410406ace0 0x7f410406d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41200cefb0 0x7f412014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f41200b16d0 0x7f41200ab8c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 --2- 172.21.15.79:0/1420830647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41200cb620 0x7f41200c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 >> 172.21.15.79:0/1420830647 conn(0x7f41200cbfc0 msgr2=0x7f412015eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:40.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.207+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 shutdown_connections 2023-06-08T03:51:40.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.208+0000 7f4127ac7700 1 -- 172.21.15.79:0/1420830647 wait complete. 2023-06-08T03:51:40.213 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' removed 2023-06-08T03:51:40.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-08T03:51:40.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.348+0000 7fb8a158d700 1 Processor -- start 2023-06-08T03:51:40.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.348+0000 7fb8a158d700 1 -- start start 2023-06-08T03:51:40.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.348+0000 7fb8a158d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb89c1586e0 0x7fb89c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.348+0000 7fb8a158d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb89c071c80 0x7fb89c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.348+0000 7fb8a158d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c0ce910 0x7fb89c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb8a158d700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89c158ab0 con 0x7fb89c071c80 2023-06-08T03:51:40.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb8a158d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89c0ced90 con 0x7fb89c1586e0 2023-06-08T03:51:40.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb8a158d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c0cc870 con 0x7fb89c0ce910 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb89c071c80 0x7fb89c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb89c071c80 0x7fb89c0ce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37256/0 (socket says 172.21.15.79:37256) 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 -- 172.21.15.79:0/60486995 learned_addr learned my addr 172.21.15.79:0/60486995 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 -- 172.21.15.79:0/60486995 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c0ce910 msgr2=0x7fb89c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89b7fe700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c0ce910 0x7fb89c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89affd700 1 --2- 172.21.15.79:0/60486995 >> v2:172.21.15.156:3300/0 conn(0x7fb89c1586e0 0x7fb89c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c0ce910 0x7fb89c0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 -- 172.21.15.79:0/60486995 >> v2:172.21.15.156:3300/0 conn(0x7fb89c1586e0 msgr2=0x7fb89c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 --2- 172.21.15.79:0/60486995 >> v2:172.21.15.156:3300/0 conn(0x7fb89c1586e0 0x7fb89c0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 -- 172.21.15.79:0/60486995 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c0ccb50 con 0x7fb89c071c80 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89b7fe700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c0ce910 0x7fb89c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89affd700 1 --2- 172.21.15.79:0/60486995 >> v2:172.21.15.156:3300/0 conn(0x7fb89c1586e0 0x7fb89c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb89a7fc700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c0ce3d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb89c06f9d0 tx=0x7fb88c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc49523872860e6 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:40.355 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb899ffb700 1 -- 172.21.15.79:0/60486995 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00b870 con 0x7fb89c071c80 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb899ffb700 1 -- 172.21.15.79:0/60486995 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00b9d0 con 0x7fb89c071c80 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.349+0000 7fb899ffb700 1 -- 172.21.15.79:0/60486995 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00bd60 con 0x7fb89c071c80 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 -- 172.21.15.79:0/60486995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 msgr2=0x7fb89c0ce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c0ce3d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb89c06f9d0 tx=0x7fb88c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 -- 172.21.15.79:0/60486995 shutdown_connections 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c0ce910 0x7fb89c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 --2- 172.21.15.79:0/60486995 >> v2:172.21.15.156:3300/0 conn(0x7fb89c1586e0 0x7fb89c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 --2- 172.21.15.79:0/60486995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c0ce3d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 -- 172.21.15.79:0/60486995 >> 172.21.15.79:0/60486995 conn(0x7fb89c069d50 msgr2=0x7fb89c06a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:40.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 -- 172.21.15.79:0/60486995 shutdown_connections 2023-06-08T03:51:40.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.350+0000 7fb8a158d700 1 -- 172.21.15.79:0/60486995 wait complete. 2023-06-08T03:51:40.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.351+0000 7fb8a158d700 1 Processor -- start 2023-06-08T03:51:40.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.351+0000 7fb8a158d700 1 -- start start 2023-06-08T03:51:40.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb8a158d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb8a158d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c0ce910 0x7fb89c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb8a158d700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c1586e0 0x7fb89c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb8a158d700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c14bad0 con 0x7fb89c071c80 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb8a158d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1596a0 con 0x7fb89c0ce910 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb8a158d700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c0ad2b0 con 0x7fb89c1586e0 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37262/0 (socket says 172.21.15.79:37262) 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 -- 172.21.15.79:0/1122522304 learned_addr learned my addr 172.21.15.79:0/1122522304 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89a7fc700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c0ce910 0x7fb89c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89b7fe700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c1586e0 0x7fb89c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 -- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c1586e0 msgr2=0x7fb89c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c1586e0 0x7fb89c0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 -- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c0ce910 msgr2=0x7fb89c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c0ce910 0x7fb89c0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.352+0000 7fb89affd700 1 -- 172.21.15.79:0/1122522304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009550 con 0x7fb89c071c80 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb89b7fe700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c1586e0 0x7fb89c0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:40.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb89affd700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c15a7d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c7f0 tx=0x7fb89000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890011800 con 0x7fb89c071c80 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890011960 con 0x7fb89c071c80 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c159940 con 0x7fb89c071c80 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890011d10 con 0x7fb89c071c80 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.353+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c159f00 con 0x7fb89c071c80 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.354+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c0c6f00 con 0x7fb89c071c80 2023-06-08T03:51:40.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.354+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb890018360 con 0x7fb89c071c80 2023-06-08T03:51:40.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.354+0000 7fb8837fe700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb88806ace0 0x7fb88806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:40.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.355+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb890089600 con 0x7fb89c071c80 2023-06-08T03:51:40.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.355+0000 7fb89a7fc700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb88806ace0 0x7fb88806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:40.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.355+0000 7fb89a7fc700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb88806ace0 0x7fb88806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb88c014040 tx=0x7fb88c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:40.363 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.359+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb890057fa0 con 0x7fb89c071c80 2023-06-08T03:51:40.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:40.543+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb89c073e90 con 0x7fb89c071c80 2023-06-08T03:51:41.208 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.204+0000 7fb8837fe700 1 -- 172.21.15.79:0/1122522304 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb890057b30 con 0x7fb89c071c80 2023-06-08T03:51:41.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb88806ace0 msgr2=0x7fb88806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:41.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb88806ace0 0x7fb88806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb88c014040 tx=0x7fb88c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:41.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 msgr2=0x7fb89c15a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:41.211 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c15a7d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb89000c7f0 tx=0x7fb89000cbb0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 shutdown_connections 2023-06-08T03:51:41.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb88806ace0 0x7fb88806d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb89c1586e0 0x7fb89c0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb89c071c80 0x7fb89c15a7d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 --2- 172.21.15.79:0/1122522304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c0ce910 0x7fb89c0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 >> 172.21.15.79:0/1122522304 conn(0x7fb89c069d50 msgr2=0x7fb89c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:41.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.208+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 shutdown_connections 2023-06-08T03:51:41.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.209+0000 7fb8a158d700 1 -- 172.21.15.79:0/1122522304 wait complete. 2023-06-08T03:51:41.214 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' removed 2023-06-08T03:51:41.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-06-08T03:51:41.349 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.347+0000 7f174f166700 1 Processor -- start 2023-06-08T03:51:41.350 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.347+0000 7f174f166700 1 -- start start 2023-06-08T03:51:41.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174f166700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1748070a60 0x7f174806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174f166700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1748070e30 0x7f174806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.351 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174f166700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f174815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174f166700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17480cac90 con 0x7f1748070a60 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174f166700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1748069d50 con 0x7f1748070e30 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174f166700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17480ccb50 con 0x7f174806aa80 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1748070a60 0x7f174806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1748070a60 0x7f174806d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37270/0 (socket says 172.21.15.79:37270) 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 -- 172.21.15.79:0/3763055079 learned_addr learned my addr 172.21.15.79:0/3763055079 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:41.352 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f1747fff700 1 --2- 172.21.15.79:0/3763055079 >> v2:172.21.15.156:3300/0 conn(0x7f1748070e30 0x7f174806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174d703700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f174815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 -- 172.21.15.79:0/3763055079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 msgr2=0x7f174815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f174815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 -- 172.21.15.79:0/3763055079 >> v2:172.21.15.156:3300/0 conn(0x7f1748070e30 msgr2=0x7f174806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 --2- 172.21.15.79:0/3763055079 >> v2:172.21.15.156:3300/0 conn(0x7f1748070e30 0x7f174806da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 -- 172.21.15.79:0/3763055079 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174806e050 con 0x7f1748070a60 2023-06-08T03:51:41.353 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f1747fff700 1 --2- 172.21.15.79:0/3763055079 >> v2:172.21.15.156:3300/0 conn(0x7f1748070e30 0x7f174806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:41.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174d703700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f174815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:41.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f174cf02700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f174806d4e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1748072140 tx=0x7f173c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c53e1a67cced05 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:41.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f17477fe700 1 -- 172.21.15.79:0/3763055079 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00b870 con 0x7f1748070a60 2023-06-08T03:51:41.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f17477fe700 1 -- 172.21.15.79:0/3763055079 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c00b9d0 con 0x7f1748070a60 2023-06-08T03:51:41.354 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.348+0000 7f17477fe700 1 -- 172.21.15.79:0/3763055079 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00bd60 con 0x7f1748070a60 2023-06-08T03:51:41.356 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 -- 172.21.15.79:0/3763055079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 msgr2=0x7f174806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f174806d4e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1748072140 tx=0x7f173c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 -- 172.21.15.79:0/3763055079 shutdown_connections 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f174815bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 --2- 172.21.15.79:0/3763055079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f174806d4e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 --2- 172.21.15.79:0/3763055079 >> v2:172.21.15.156:3300/0 conn(0x7f1748070e30 0x7f174806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 -- 172.21.15.79:0/3763055079 >> 172.21.15.79:0/3763055079 conn(0x7f17480ceca0 msgr2=0x7f17480cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:41.357 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 -- 172.21.15.79:0/3763055079 shutdown_connections 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.349+0000 7f174f166700 1 -- 172.21.15.79:0/3763055079 wait complete. 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.350+0000 7f174f166700 1 Processor -- start 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.350+0000 7f174f166700 1 -- start start 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174f166700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f17481f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174f166700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f17480aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174f166700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1748070e30 0x7f17480aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.358 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174f166700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174814e760 con 0x7f1748070a60 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174f166700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748150f60 con 0x7f1748070e30 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174f166700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174814dc90 con 0x7f174806aa80 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f17480aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f17480aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37278/0 (socket says 172.21.15.79:37278) 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 -- 172.21.15.79:0/2445200581 learned_addr learned my addr 172.21.15.79:0/2445200581 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174cf02700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f17481f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.359 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174d703700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1748070e30 0x7f17480aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 -- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 msgr2=0x7f17481f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f17481f1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 -- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1748070e30 msgr2=0x7f17480aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1748070e30 0x7f17480aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f1747fff700 1 -- 172.21.15.79:0/2445200581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c009550 con 0x7f1748070a60 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.351+0000 7f174cf02700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f17481f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.352+0000 7f1747fff700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f17480aea50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f173400e760 tx=0x7f173400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:41.360 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.352+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734013610 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.352+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1734013770 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.352+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17481513f0 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.352+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734013b20 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.352+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174814e0a0 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.353+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174814e8a0 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.353+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1734013c80 con 0x7f1748070a60 2023-06-08T03:51:41.361 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.354+0000 7f17457fa700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f172c06ace0 0x7f172c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:41.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.354+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f1734088fe0 con 0x7f1748070a60 2023-06-08T03:51:41.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.354+0000 7f174cf02700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f172c06ace0 0x7f172c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:41.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.355+0000 7f174cf02700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f172c06ace0 0x7f172c06d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f173c009130 tx=0x7f173c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:41.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.358+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f174814e8a0 con 0x7f1748070a60 2023-06-08T03:51:41.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:41.546+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f174814e8a0 con 0x7f1748070a60 2023-06-08T03:51:42.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.213+0000 7f17457fa700 1 -- 172.21.15.79:0/2445200581 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v29) v1 ==== 178+0+0 (secure 0 0 0) 0x7f174814e8a0 con 0x7f1748070a60 2023-06-08T03:51:42.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f172c06ace0 msgr2=0x7f172c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:42.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f172c06ace0 0x7f172c06d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f173c009130 tx=0x7f173c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:42.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 msgr2=0x7f17480aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:42.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f17480aea50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f173400e760 tx=0x7f173400eb20 comp rx=0 tx=0).stop 2023-06-08T03:51:42.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 shutdown_connections 2023-06-08T03:51:42.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1748070e30 0x7f17480aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f172c06ace0 0x7f172c06d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f174806aa80 0x7f17481f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 --2- 172.21.15.79:0/2445200581 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1748070a60 0x7f17480aea50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 >> 172.21.15.79:0/2445200581 conn(0x7f17480ceca0 msgr2=0x7f17480726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:42.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.217+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 shutdown_connections 2023-06-08T03:51:42.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.218+0000 7f174f166700 1 -- 172.21.15.79:0/2445200581 wait complete. 2023-06-08T03:51:42.223 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data_ec' removed 2023-06-08T03:51:42.233 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-08T03:51:42.234 INFO:tasks.ceph.mds.a:Stopping old one... 2023-06-08T03:51:42.234 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-08T03:51:42.234 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:42.235 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-08T03:51:42.235 INFO:tasks.ceph.mds.c:Stopping old one... 2023-06-08T03:51:42.235 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-08T03:51:42.235 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:42.235 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-08T03:51:42.236 INFO:tasks.ceph.mds.b:Stopping old one... 2023-06-08T03:51:42.236 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-08T03:51:42.236 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:42.236 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-08T03:51:42.236 INFO:tasks.ceph.mds.d:Stopping old one... 2023-06-08T03:51:42.237 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-08T03:51:42.237 INFO:teuthology.orchestra.run:waiting for 300 2023-06-08T03:51:42.243 INFO:tasks.ceph.mds.a:Stopped 2023-06-08T03:51:42.244 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-08T03:51:42.244 INFO:tasks.ceph.mds.c:Stopped 2023-06-08T03:51:42.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-08T03:51:42.247 INFO:tasks.ceph.mds.b:Stopped 2023-06-08T03:51:42.247 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-08T03:51:42.248 INFO:tasks.ceph.mds.d:Stopped 2023-06-08T03:51:42.248 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-08T03:51:42.285 INFO:tasks.ceph.mds.a:Started 2023-06-08T03:51:42.286 DEBUG:teuthology.parallel:result is None 2023-06-08T03:51:42.286 INFO:tasks.ceph.mds.c:Started 2023-06-08T03:51:42.287 DEBUG:teuthology.parallel:result is None 2023-06-08T03:51:42.289 INFO:tasks.ceph.mds.b:Started 2023-06-08T03:51:42.289 DEBUG:teuthology.parallel:result is None 2023-06-08T03:51:42.290 INFO:tasks.ceph.mds.d:Started 2023-06-08T03:51:42.290 DEBUG:teuthology.parallel:result is None 2023-06-08T03:51:42.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-06-08T03:51:42.364 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2023-06-08T03:51:42.364 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2023-06-08T03:51:42.365 INFO:tasks.ceph.mds.c.smithi079.stderr:2023-06-08T03:51:42.362+0000 7f7ffbae3700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-06-08T03:51:42.375 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-06-08T03:51:42.375 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-06-08T03:51:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.423+0000 7f464702f700 1 Processor -- start 2023-06-08T03:51:42.426 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.423+0000 7f464702f700 1 -- start start 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f464702f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464015ffa0 0x7f4640160370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f464702f700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f464006c750 0x7f46401608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f464702f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464006ae60 con 0x7f46400b29a0 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f464702f700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464015d430 con 0x7f464006c750 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f464702f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464015d710 con 0x7f464015ffa0 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f464006c750 0x7f46401608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:42.428 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f464006c750 0x7f46401608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37312/0 (socket says 172.21.15.79:37312) 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 -- 172.21.15.79:0/976040333 learned_addr learned my addr 172.21.15.79:0/976040333 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f4644dcb700 1 --2- 172.21.15.79:0/976040333 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464015ffa0 0x7f4640160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 -- 172.21.15.79:0/976040333 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464015ffa0 msgr2=0x7f4640160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 --2- 172.21.15.79:0/976040333 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464015ffa0 0x7f4640160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 -- 172.21.15.79:0/976040333 >> v1:172.21.15.156:6789/0 conn(0x7f46400b29a0 legacy=0x7f46400cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 -- 172.21.15.79:0/976040333 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46400ce5c0 con 0x7f464006c750 2023-06-08T03:51:42.429 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.424+0000 7f46455cc700 1 --2- 172.21.15.79:0/976040333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464006c750 0x7f46401608b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f46380098f0 tx=0x7f4638009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcf648cfe69010d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:42.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f463f7fe700 1 -- 172.21.15.79:0/976040333 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800b8a0 con 0x7f464006c750 2023-06-08T03:51:42.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f463f7fe700 1 -- 172.21.15.79:0/976040333 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463800ba00 con 0x7f464006c750 2023-06-08T03:51:42.430 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f463f7fe700 1 -- 172.21.15.79:0/976040333 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800bd90 con 0x7f464006c750 2023-06-08T03:51:42.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 -- 172.21.15.79:0/976040333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464006c750 msgr2=0x7f46401608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:42.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 --2- 172.21.15.79:0/976040333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464006c750 0x7f46401608b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f46380098f0 tx=0x7f4638009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:42.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 -- 172.21.15.79:0/976040333 shutdown_connections 2023-06-08T03:51:42.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 --2- 172.21.15.79:0/976040333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464006c750 0x7f46401608b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.432 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 --2- 172.21.15.79:0/976040333 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464015ffa0 0x7f4640160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 -- 172.21.15.79:0/976040333 >> 172.21.15.79:0/976040333 conn(0x7f4640069d50 msgr2=0x7f464006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 -- 172.21.15.79:0/976040333 shutdown_connections 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.425+0000 7f464702f700 1 -- 172.21.15.79:0/976040333 wait complete. 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.427+0000 7f464702f700 1 Processor -- start 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.427+0000 7f464702f700 1 -- start start 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f464702f700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464006c750 0x7f4640159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f464702f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464015ffa0 0x7f46400c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:42.433 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f464702f700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 0x7f46400af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f464702f700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4640158bc0 con 0x7f464015a960 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f464702f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400c5fa0 con 0x7f464015ffa0 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f464702f700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46400c6280 con 0x7f464006c750 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 0x7f46400af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 0x7f46400af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37318/0 (socket says 172.21.15.79:37318) 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 -- 172.21.15.79:0/3916484360 learned_addr learned my addr 172.21.15.79:0/3916484360 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:42.434 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f4644dcb700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464006c750 0x7f4640159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f463ffff700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464015ffa0 0x7f46400c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 -- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464006c750 msgr2=0x7f4640159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464006c750 0x7f4640159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 -- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464015ffa0 msgr2=0x7f46400c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464015ffa0 0x7f46400c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.428+0000 7f46455cc700 1 -- 172.21.15.79:0/3916484360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009550 con 0x7f464015a960 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.429+0000 7f46455cc700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 0x7f46400af230 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f46400d1770 tx=0x7f4638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:42.435 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.429+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638004510 con 0x7f464015a960 2023-06-08T03:51:42.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.429+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46380036a0 con 0x7f464015a960 2023-06-08T03:51:42.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.429+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4640158570 con 0x7f464015a960 2023-06-08T03:51:42.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.429+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638003a50 con 0x7f464015a960 2023-06-08T03:51:42.436 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.429+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4640159590 con 0x7f464015a960 2023-06-08T03:51:42.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.430+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640159720 con 0x7f464015a960 2023-06-08T03:51:42.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.430+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4638021070 con 0x7f464015a960 2023-06-08T03:51:42.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.431+0000 7f463d7fa700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f462c06ace0 0x7f462c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:42.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.431+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f463808a760 con 0x7f464015a960 2023-06-08T03:51:42.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.431+0000 7f4644dcb700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f462c06ace0 0x7f462c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:42.437 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.431+0000 7f4644dcb700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f462c06ace0 0x7f462c06d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4630003b30 tx=0x7f4630009830 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:42.438 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.435+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f46380559e0 con 0x7f464015a960 2023-06-08T03:51:42.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:42.635+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4640073e90 con 0x7f464015a960 2023-06-08T03:51:43.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.399+0000 7f463d7fa700 1 -- 172.21.15.79:0/3916484360 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4638059000 con 0x7f464015a960 2023-06-08T03:51:43.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f462c06ace0 msgr2=0x7f462c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f462c06ace0 0x7f462c06d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4630003b30 tx=0x7f4630009830 comp rx=0 tx=0).stop 2023-06-08T03:51:43.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 msgr2=0x7f46400af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 0x7f46400af230 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f46400d1770 tx=0x7f4638004750 comp rx=0 tx=0).stop 2023-06-08T03:51:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 shutdown_connections 2023-06-08T03:51:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f464015a960 0x7f46400af230 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f462c06ace0 0x7f462c06d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f464006c750 0x7f4640159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 --2- 172.21.15.79:0/3916484360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f464015ffa0 0x7f46400c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.401+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 >> 172.21.15.79:0/3916484360 conn(0x7f4640069d50 msgr2=0x7f4640157830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:43.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.402+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 shutdown_connections 2023-06-08T03:51:43.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.402+0000 7f464702f700 1 -- 172.21.15.79:0/3916484360 wait complete. 2023-06-08T03:51:43.408 INFO:teuthology.orchestra.run.smithi079.stderr: removed all blocklist entries 2023-06-08T03:51:43.417 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-06-08T03:51:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.534+0000 7fcffbe0e700 1 Processor -- start 2023-06-08T03:51:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.535+0000 7fcffbe0e700 1 -- start start 2023-06-08T03:51:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.535+0000 7fcffbe0e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 0x7fcff40cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.535+0000 7fcffbe0e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcff40cd7a0 0x7fcff40b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.535+0000 7fcffbe0e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff40ce250 con 0x7fcff40ccb30 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.535+0000 7fcffbe0e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff40afeb0 con 0x7fcff40cd7a0 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.535+0000 7fcffbe0e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40ce4a0 con 0x7fcff415d7a0 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff9baa700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 0x7fcff40cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcff40cd7a0 0x7fcff40b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff93a9700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fcff40ccb30 0x7fcff40cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:34732/0 (socket says 172.21.15.79:34732) 2023-06-08T03:51:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcff40cd7a0 0x7fcff40b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:60224/0 (socket says 172.21.15.79:60224) 2023-06-08T03:51:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff93a9700 1 -- 172.21.15.79:0/27953360 learned_addr learned my addr 172.21.15.79:0/27953360 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff9baa700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 0x7fcff40cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:51648/0 (socket says 172.21.15.79:51648) 2023-06-08T03:51:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 -- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 msgr2=0x7fcff40cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 --2- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 0x7fcff40cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 -- 172.21.15.79:0/27953360 >> v1:172.21.15.156:6789/0 conn(0x7fcff40ccb30 legacy=0x7fcff40cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 -- 172.21.15.79:0/27953360 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40ce780 con 0x7fcff40cd7a0 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff9baa700 1 --2- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 0x7fcff40cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcffa3ab700 1 --2- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff40cd7a0 0x7fcff40b0340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009d90 tx=0x7fcfec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619d88dd54945341 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff8ba8700 1 -- 172.21.15.79:0/27953360 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00e070 con 0x7fcff40cd7a0 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff8ba8700 1 -- 172.21.15.79:0/27953360 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec00bb20 con 0x7fcff40cd7a0 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.536+0000 7fcff8ba8700 1 -- 172.21.15.79:0/27953360 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec012430 con 0x7fcff40cd7a0 2023-06-08T03:51:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 -- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff40cd7a0 msgr2=0x7fcff40b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff40cd7a0 0x7fcff40b0340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009d90 tx=0x7fcfec00b330 comp rx=0 tx=0).stop 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 -- 172.21.15.79:0/27953360 shutdown_connections 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff40cd7a0 0x7fcff40b0340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/27953360 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff415d7a0 0x7fcff40cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 -- 172.21.15.79:0/27953360 >> 172.21.15.79:0/27953360 conn(0x7fcff4099960 msgr2=0x7fcff40d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 -- 172.21.15.79:0/27953360 shutdown_connections 2023-06-08T03:51:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.537+0000 7fcffbe0e700 1 -- 172.21.15.79:0/27953360 wait complete. 2023-06-08T03:51:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.538+0000 7fcffbe0e700 1 Processor -- start 2023-06-08T03:51:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.538+0000 7fcffbe0e700 1 -- start start 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffbe0e700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff40ccb30 0x7fcff40b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffbe0e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 0x7fcff4152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffbe0e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff415d7a0 0x7fcff40d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffbe0e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4157030 con 0x7fcff40cd7a0 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffbe0e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41f0800 con 0x7fcff415d7a0 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffbe0e700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40b6590 con 0x7fcff40ccb30 2023-06-08T03:51:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 0x7fcff4152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 0x7fcff4152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37332/0 (socket says 172.21.15.79:37332) 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 -- 172.21.15.79:0/1006970635 learned_addr learned my addr 172.21.15.79:0/1006970635 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff9baa700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff40ccb30 0x7fcff40b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcffa3ab700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff415d7a0 0x7fcff40d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 -- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff40ccb30 msgr2=0x7fcff40b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff40ccb30 0x7fcff40b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 -- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff415d7a0 msgr2=0x7fcff40d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff415d7a0 0x7fcff40d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 -- 172.21.15.79:0/1006970635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe80094a0 con 0x7fcff40cd7a0 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.539+0000 7fcff93a9700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 0x7fcff4152ab0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcff000c4b0 tx=0x7fcff000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.540+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0005d70 con 0x7fcff40cd7a0 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.540+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000ce80 con 0x7fcff40cd7a0 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.540+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff00072d0 con 0x7fcff40cd7a0 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.540+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec009a70 con 0x7fcff40cd7a0 2023-06-08T03:51:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.540+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff41efe60 con 0x7fcff40cd7a0 2023-06-08T03:51:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.541+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff40cdfb0 con 0x7fcff40cd7a0 2023-06-08T03:51:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.541+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcff0007430 con 0x7fcff40cd7a0 2023-06-08T03:51:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.542+0000 7fcfe67fc700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcfd806ace0 0x7fcfd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.542+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcff0088390 con 0x7fcff40cd7a0 2023-06-08T03:51:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.542+0000 7fcff9baa700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcfd806ace0 0x7fcfd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.542+0000 7fcff9baa700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcfd806ace0 0x7fcfd806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009d70 tx=0x7fcfe8009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.546+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcff0053790 con 0x7fcff40cd7a0 2023-06-08T03:51:43.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.807+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcff41f0940 con 0x7fcff40cd7a0 2023-06-08T03:51:43.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.808+0000 7fcfe67fc700 1 -- 172.21.15.79:0/1006970635 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fcff0056db0 con 0x7fcff40cd7a0 2023-06-08T03:51:43.811 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:43.811 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:51:43.811 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_dump": [ 2023-06-08T03:51:43.811 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.812 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.a", 2023-06-08T03:51:43.812 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FkRZDBEhAAN5RLnYkm+w9jFxGSf8TjeQ==", 2023-06-08T03:51:43.812 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.812 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.812 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-08T03:51:43.813 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-08T03:51:43.813 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.813 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.b", 2023-06-08T03:51:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FkRujWHhAAP6t5Gy6nJqvbJflDSdvM9Q==", 2023-06-08T03:51:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.814 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.815 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.c", 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4Fk4cf2GRAAyRD7ooUmmArn9eNTBu9p1Q==", 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.816 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.d", 2023-06-08T03:51:43.817 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FkqmMJJhAAO3QcZ8jcPmU9RndPFbc2jg==", 2023-06-08T03:51:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-08T03:51:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-08T03:51:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.818 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.0", 2023-06-08T03:51:43.819 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDST4FkTVJmARAAwhbt9gcKhJmxNMvLCCZNgA==", 2023-06-08T03:51:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.820 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.821 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.1", 2023-06-08T03:51:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDVT4Fk1ICyARAARd9komdmzhtLzhWG1Cwa9g==", 2023-06-08T03:51:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.822 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.2", 2023-06-08T03:51:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDYT4FkjaLQABAAjXntGlmvUt4Eixrlsqv6pg==", 2023-06-08T03:51:43.823 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.824 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.3", 2023-06-08T03:51:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDaT4FkfZmZOxAAYEUB+u4uXJPWRr+iCIEzfw==", 2023-06-08T03:51:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.825 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.4", 2023-06-08T03:51:43.826 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDfT4FkLEajDhAAFaS4OndZz6gW+9pJinUziw==", 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.827 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.828 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.5", 2023-06-08T03:51:43.828 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDiT4Fkh5tEDhAANLbWNGFTbafQ3h3FspueuA==", 2023-06-08T03:51:43.828 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.829 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.829 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.829 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.829 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.829 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.829 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.830 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.6", 2023-06-08T03:51:43.830 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDlT4Fk+GtZDRAAd72KpRrYlvYq5OHVbbrNbg==", 2023-06-08T03:51:43.830 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.830 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.831 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.831 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.831 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.831 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.832 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.832 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.7", 2023-06-08T03:51:43.832 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDoT4FkbWlyDBAAtLYK0i2yjivtP0+2XgRY9Q==", 2023-06-08T03:51:43.832 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.833 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.0", 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FknZFgKhAA1Hf83RGBLm0IVJwZZeXrNg==", 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.1", 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4Fk31fyMBAAa2EcGDOxAywwlFcdTFwhPA==", 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-08T03:51:43.835 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.2", 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FkqH+rLRAAbpuUe2nkVYasrtpRL1jcIA==", 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.836 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.3", 2023-06-08T03:51:43.837 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FkXmCmNhAAOyOsxsYfveZPi1hP/apNjw==", 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.838 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.admin", 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDNT4FkcImCMBAAXrG1xKI+tIRnnC0V5ZRh7A==", 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-mds", 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDuT4FkDDwFCRAAoDuoN+Cn+Sulgy7a8UpL8g==", 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-mds" 2023-06-08T03:51:43.840 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-mgr", 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDuT4Fkn0QFCRAAQxFgHEaDg7lUcXAGmix8xQ==", 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-mgr" 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.841 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-osd", 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDuT4FkMUwFCRAAqqJhzrVgBKoeNsEJLey4OQ==", 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-osd" 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.842 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.843 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.843 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rbd", 2023-06-08T03:51:43.843 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDuT4Fkr1UFCRAAChdD2dSygKunW2YI5V8eKQ==", 2023-06-08T03:51:43.844 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.844 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rbd" 2023-06-08T03:51:43.844 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.844 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDuT4FkRl0FCRAAsKxAxE8ZyDuNH+6vpskDpw==", 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rgw", 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDuT4Fk7mQFCRAAMSG41wEknYemMZ4xQsB/eg==", 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rgw" 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.846 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.847 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mgr.x", 2023-06-08T03:51:43.847 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4Fkc+5vDxAAFCxhBuMs5zg/ox2OeO3i8w==", 2023-06-08T03:51:43.847 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.847 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2023-06-08T03:51:43.847 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile mgr", 2023-06-08T03:51:43.848 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.848 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:43.848 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:43.849 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mgr.y", 2023-06-08T03:51:43.849 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQDOT4FkDgb5CRAAzNxBhP5cwvsr6T+vPZgjbQ==", 2023-06-08T03:51:43.849 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2023-06-08T03:51:43.849 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2023-06-08T03:51:43.849 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile mgr", 2023-06-08T03:51:43.849 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2023-06-08T03:51:43.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.850 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:43.850 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2023-06-08T03:51:43.850 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:51:43.852 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcfd806ace0 msgr2=0x7fcfd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcfd806ace0 0x7fcfd806d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009d70 tx=0x7fcfe8009210 comp rx=0 tx=0).stop 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 msgr2=0x7fcff4152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 0x7fcff4152ab0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcff000c4b0 tx=0x7fcff000c7c0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 shutdown_connections 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcff415d7a0 0x7fcff40d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fcfd806ace0 0x7fcfd806d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.853 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcff40ccb30 0x7fcff40b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 --2- 172.21.15.79:0/1006970635 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcff40cd7a0 0x7fcff4152ab0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 >> 172.21.15.79:0/1006970635 conn(0x7fcff4099960 msgr2=0x7fcff406faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:43.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 shutdown_connections 2023-06-08T03:51:43.854 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.810+0000 7fcffbe0e700 1 -- 172.21.15.79:0/1006970635 wait complete. 2023-06-08T03:51:43.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-06-08T03:51:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.972+0000 7f2370eef700 1 Processor -- start 2023-06-08T03:51:43.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.972+0000 7f2370eef700 1 -- start start 2023-06-08T03:51:43.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f2370eef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 0x7f236c071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f2370eef700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0cd140 con 0x7f236c069d50 2023-06-08T03:51:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f2370eef700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c071110 con 0x7f236c15c1f0 2023-06-08T03:51:43.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f2370eef700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c071be0 con 0x7f236c06d8a0 2023-06-08T03:51:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f236ad9d700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f236c069d50 0x7f236c0cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60818/0 (socket says 172.21.15.79:60818) 2023-06-08T03:51:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f236ad9d700 1 -- 172.21.15.79:0/2516942528 learned_addr learned my addr 172.21.15.79:0/2516942528 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111156252 0 0) 0x7f236c0cd140 con 0x7f236c069d50 2023-06-08T03:51:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f2369d9b700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 0x7f236c071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.973+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23540035b0 con 0x7f236c069d50 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1033886132 0 0) 0x7f23540035b0 con 0x7f236c069d50 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 msgr2=0x7f236c071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 0x7f236c071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 >> v1:172.21.15.156:6789/0 conn(0x7f236c15c1f0 legacy=0x7f236c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0cd390 con 0x7f236c069d50 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f23640019f0 con 0x7f236c069d50 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 >> v1:172.21.15.79:6789/0 conn(0x7f236c069d50 legacy=0x7f236c0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2354003a30 0x7f2354003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354004850 0x7f2354004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 0x7f2354005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c071be0 con 0x7f2354003a30 2023-06-08T03:51:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235400bb40 con 0x7f2354004850 2023-06-08T03:51:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235400bd00 con 0x7f236c06d8a0 2023-06-08T03:51:43.982 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2354003a30 0x7f2354003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f2369d9b700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354004850 0x7f2354004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236ad9d700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 0x7f2354005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 -- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 msgr2=0x7f2354005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:43.983 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c06d8a0 0x7f2354005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 -- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354004850 msgr2=0x7f2354004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:43.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354004850 0x7f2354004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 -- 172.21.15.79:0/2516942528 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23540035b0 con 0x7f2354003a30 2023-06-08T03:51:43.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f2369d9b700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354004850 0x7f2354004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:43.984 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236a59c700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2354003a30 0x7f2354003e00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f23600018d0 tx=0x7f2354005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3090b145948a83b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:43.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012620 con 0x7f2354003a30 2023-06-08T03:51:43.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23600127a0 con 0x7f2354003a30 2023-06-08T03:51:43.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.974+0000 7f236959a700 1 -- 172.21.15.79:0/2516942528 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012b50 con 0x7f2354003a30 2023-06-08T03:51:43.985 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f2370eef700 1 -- 172.21.15.79:0/2516942528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2354003a30 msgr2=0x7f2354003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f2370eef700 1 --2- 172.21.15.79:0/2516942528 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2354003a30 0x7f2354003e00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f23600018d0 tx=0x7f2354005880 comp rx=0 tx=0).stop 2023-06-08T03:51:43.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f236ad9d700 1 -- 172.21.15.79:0/2516942528 reap_dead start 2023-06-08T03:51:43.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f2370eef700 1 -- 172.21.15.79:0/2516942528 shutdown_connections 2023-06-08T03:51:43.986 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f2370eef700 1 -- 172.21.15.79:0/2516942528 >> 172.21.15.79:0/2516942528 conn(0x7f236c0b1d30 msgr2=0x7f236c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:43.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f2370eef700 1 -- 172.21.15.79:0/2516942528 shutdown_connections 2023-06-08T03:51:43.987 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.975+0000 7f2370eef700 1 -- 172.21.15.79:0/2516942528 wait complete. 2023-06-08T03:51:43.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.976+0000 7f2370eef700 1 Processor -- start 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.976+0000 7f2370eef700 1 -- start start 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2370eef700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354003a30 0x7f236c1f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2370eef700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 0x7f236c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2370eef700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c1f0ef0 0x7f236c0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2370eef700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1f1eb0 con 0x7f236c1564b0 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2370eef700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15db90 con 0x7f2354003a30 2023-06-08T03:51:43.990 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2370eef700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c157380 con 0x7f236c1f0ef0 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 0x7f236c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 0x7f236c156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37354/0 (socket says 172.21.15.79:37354) 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f2369d9b700 1 -- 172.21.15.79:0/3326463929 learned_addr learned my addr 172.21.15.79:0/3326463929 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.977+0000 7f236ad9d700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c1f0ef0 0x7f236c0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f236a59c700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354003a30 0x7f236c1f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2369d9b700 1 -- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c1f0ef0 msgr2=0x7f236c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.991 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2369d9b700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c1f0ef0 0x7f236c0cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2369d9b700 1 -- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354003a30 msgr2=0x7f236c1f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2369d9b700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354003a30 0x7f236c1f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2369d9b700 1 -- 172.21.15.79:0/3326463929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c1575b0 con 0x7f236c1564b0 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2369d9b700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 0x7f236c156e40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f235c011900 tx=0x7f235c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c004220 con 0x7f236c1564b0 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c004380 con 0x7f236c1564b0 2023-06-08T03:51:43.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c157890 con 0x7f236c1564b0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0036a0 con 0x7f236c1564b0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.978+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1f1ad0 con 0x7f236c1564b0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.979+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c073e90 con 0x7f236c1564b0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.979+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f235c003800 con 0x7f236c1564b0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.980+0000 7f235affd700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f234c06ace0 0x7f234c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.980+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f235c089590 con 0x7f236c1564b0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.980+0000 7f236a59c700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f234c06ace0 0x7f234c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:43.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.980+0000 7f236a59c700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f234c06ace0 0x7f234c06d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23600018d0 tx=0x7f2360005040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:43.994 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:43.984+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f235c0548e0 con 0x7f236c1564b0 2023-06-08T03:51:44.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.248+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f236c06ed70 con 0x7f236c1564b0 2023-06-08T03:51:44.254 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.251+0000 7f235affd700 1 -- 172.21.15.79:0/3326463929 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f235c057f00 con 0x7f236c1564b0 2023-06-08T03:51:44.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f234c06ace0 msgr2=0x7f234c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f234c06ace0 0x7f234c06d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f23600018d0 tx=0x7f2360005040 comp rx=0 tx=0).stop 2023-06-08T03:51:44.257 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 msgr2=0x7f236c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 0x7f236c156e40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f235c011900 tx=0x7f235c011cc0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 shutdown_connections 2023-06-08T03:51:44.258 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f234c06ace0 0x7f234c06d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f236c1f0ef0 0x7f236c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2354003a30 0x7f236c1f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 --2- 172.21.15.79:0/3326463929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f236c1564b0 0x7f236c156e40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.259 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 >> 172.21.15.79:0/3326463929 conn(0x7f236c0b1d30 msgr2=0x7f236c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:44.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 shutdown_connections 2023-06-08T03:51:44.260 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.253+0000 7f2370eef700 1 -- 172.21.15.79:0/3326463929 wait complete. 2023-06-08T03:51:44.268 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-06-08T03:51:44.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.386+0000 7f5de34b7700 1 Processor -- start 2023-06-08T03:51:44.390 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de34b7700 1 -- start start 2023-06-08T03:51:44.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de34b7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5ddc15c1f0 0x7f5ddc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de34b7700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc0cd140 con 0x7f5ddc069d50 2023-06-08T03:51:44.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de34b7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc071110 con 0x7f5ddc071fe0 2023-06-08T03:51:44.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de34b7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc071be0 con 0x7f5ddc15c1f0 2023-06-08T03:51:44.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de1a54700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5ddc069d50 0x7f5ddc0cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60840/0 (socket says 172.21.15.79:60840) 2023-06-08T03:51:44.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.387+0000 7f5de1a54700 1 -- 172.21.15.79:0/517147176 learned_addr learned my addr 172.21.15.79:0/517147176 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:44.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 --2- 172.21.15.79:0/517147176 >> v2:172.21.15.156:3300/0 conn(0x7f5ddc15c1f0 0x7f5ddc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928717338 0 0) 0x7f5ddc0cd140 con 0x7f5ddc069d50 2023-06-08T03:51:44.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dc80035b0 con 0x7f5ddc069d50 2023-06-08T03:51:44.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1483144714 0 0) 0x7f5dc80035b0 con 0x7f5ddc069d50 2023-06-08T03:51:44.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 >> v2:172.21.15.156:3300/0 conn(0x7f5ddc15c1f0 msgr2=0x7f5ddc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:44.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 --2- 172.21.15.79:0/517147176 >> v2:172.21.15.156:3300/0 conn(0x7f5ddc15c1f0 0x7f5ddc06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 >> v1:172.21.15.156:6789/0 conn(0x7f5ddc071fe0 legacy=0x7f5ddc0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0cd390 con 0x7f5ddc069d50 2023-06-08T03:51:44.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f5dd40019f0 con 0x7f5ddc069d50 2023-06-08T03:51:44.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 >> v1:172.21.15.79:6789/0 conn(0x7f5ddc069d50 legacy=0x7f5ddc0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.403 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8003a00 0x7f5dc8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc8004850 0x7f5dc8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8005320 0x7f5dc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc071be0 con 0x7f5dc8003a00 2023-06-08T03:51:44.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc800c250 con 0x7f5dc8004850 2023-06-08T03:51:44.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc800c410 con 0x7f5dc8005320 2023-06-08T03:51:44.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8003a00 0x7f5dc8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de0a52700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc8004850 0x7f5dc8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1a54700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8005320 0x7f5dc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 -- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8005320 msgr2=0x7f5dc800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:44.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8005320 0x7f5dc800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 -- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc8004850 msgr2=0x7f5dc8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:44.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dc8004850 0x7f5dc8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1253700 1 -- 172.21.15.79:0/517147176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80035b0 con 0x7f5dc8003a00 2023-06-08T03:51:44.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.388+0000 7f5de1a54700 1 -- 172.21.15.79:0/517147176 reap_dead start 2023-06-08T03:51:44.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de1253700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8003a00 0x7f5dc8003e10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009b60 tx=0x7f5dc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721e89dc9d9e74e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:44.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00e6c0 con 0x7f5dc8003a00 2023-06-08T03:51:44.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc00e820 con 0x7f5dc8003a00 2023-06-08T03:51:44.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5dd3fff700 1 -- 172.21.15.79:0/517147176 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00ebb0 con 0x7f5dc8003a00 2023-06-08T03:51:44.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 -- 172.21.15.79:0/517147176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8003a00 msgr2=0x7f5dc8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8003a00 0x7f5dc8003e10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009b60 tx=0x7f5dc8005880 comp rx=0 tx=0).stop 2023-06-08T03:51:44.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 -- 172.21.15.79:0/517147176 shutdown_connections 2023-06-08T03:51:44.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 --2- 172.21.15.79:0/517147176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5dc8003a00 0x7f5dc8003e10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.409 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 -- 172.21.15.79:0/517147176 >> 172.21.15.79:0/517147176 conn(0x7f5ddc0b1d30 msgr2=0x7f5ddc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:44.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 -- 172.21.15.79:0/517147176 shutdown_connections 2023-06-08T03:51:44.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.389+0000 7f5de34b7700 1 -- 172.21.15.79:0/517147176 wait complete. 2023-06-08T03:51:44.410 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.390+0000 7f5de34b7700 1 Processor -- start 2023-06-08T03:51:44.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.390+0000 7f5de34b7700 1 -- start start 2023-06-08T03:51:44.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de34b7700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8003a00 0x7f5ddc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de34b7700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 0x7f5ddc1e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.411 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de34b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ddc0c4d80 0x7f5ddc0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de34b7700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0c5730 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de34b7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0c6dc0 con 0x7f5ddc0c4d80 2023-06-08T03:51:44.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de34b7700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0b4370 con 0x7f5dc8003a00 2023-06-08T03:51:44.412 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de0a52700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 0x7f5ddc1e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de0a52700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 0x7f5ddc1e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37374/0 (socket says 172.21.15.79:37374) 2023-06-08T03:51:44.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de0a52700 1 -- 172.21.15.79:0/2169172433 learned_addr learned my addr 172.21.15.79:0/2169172433 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:44.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de1253700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8003a00 0x7f5ddc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.413 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.391+0000 7f5de1a54700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ddc0c4d80 0x7f5ddc0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de0a52700 1 -- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8003a00 msgr2=0x7f5ddc15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de0a52700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8003a00 0x7f5ddc15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de0a52700 1 -- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ddc0c4d80 msgr2=0x7f5ddc0c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.414 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de0a52700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ddc0c4d80 0x7f5ddc0c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de0a52700 1 -- 172.21.15.79:0/2169172433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0b4090 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de0a52700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 0x7f5ddc1e5f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800e550 tx=0x7f5dd800e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:44.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80052e0 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.415 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ddc0c6770 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd8005440 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003880 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.416 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.392+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc15afc0 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.393+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc073e90 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.393+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5dd80039e0 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.394+0000 7f5dd1ffb700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dc006ace0 0x7f5dc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.417 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.394+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5dd8088920 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.394+0000 7f5de1253700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dc006ace0 0x7f5dc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.394+0000 7f5de1253700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dc006ace0 0x7f5dc006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0090b0 tx=0x7f5dcc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:44.418 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.398+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5dd8053ca0 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.664 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.661+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5ddc0c6f00 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.667 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.665+0000 7f5dd1ffb700 1 -- 172.21.15.79:0/2169172433 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5dd80572c0 con 0x7f5ddc1e5ad0 2023-06-08T03:51:44.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dc006ace0 msgr2=0x7f5dc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dc006ace0 0x7f5dc006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0090b0 tx=0x7f5dcc008030 comp rx=0 tx=0).stop 2023-06-08T03:51:44.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 msgr2=0x7f5ddc1e5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.671 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 0x7f5ddc1e5f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800e550 tx=0x7f5dd800e910 comp rx=0 tx=0).stop 2023-06-08T03:51:44.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 shutdown_connections 2023-06-08T03:51:44.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ddc0c4d80 0x7f5ddc0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.672 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f5dc006ace0 0x7f5dc006d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dc8003a00 0x7f5ddc15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 --2- 172.21.15.79:0/2169172433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ddc1e5ad0 0x7f5ddc1e5f40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 >> 172.21.15.79:0/2169172433 conn(0x7f5ddc0b1d30 msgr2=0x7f5ddc06a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:44.673 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 shutdown_connections 2023-06-08T03:51:44.674 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.667+0000 7f5de34b7700 1 -- 172.21.15.79:0/2169172433 wait complete. 2023-06-08T03:51:44.682 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-06-08T03:51:44.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.799+0000 7fd104515700 1 Processor -- start 2023-06-08T03:51:44.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.799+0000 7fd104515700 1 -- start start 2023-06-08T03:51:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd104515700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd0fc15c280 0x7fd0fc0d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd104515700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc0cdaa0 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd104515700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc0cdd80 con 0x7fd0fc0ceb40 2023-06-08T03:51:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd104515700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc0ce060 con 0x7fd0fc15c280 2023-06-08T03:51:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd1022b1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd0fc15c280 0x7fd0fc0d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd102ab2700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd0fc0d0b80 0x7fd0fc0af4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60860/0 (socket says 172.21.15.79:60860) 2023-06-08T03:51:44.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd102ab2700 1 -- 172.21.15.79:0/2498984867 learned_addr learned my addr 172.21.15.79:0/2498984867 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd1022b1700 1 -- 172.21.15.79:0/2498984867 >> v1:172.21.15.156:6789/0 conn(0x7fd0fc0ceb40 legacy=0x7fd0fc0d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.800+0000 7fd1022b1700 1 -- 172.21.15.79:0/2498984867 >> v1:172.21.15.79:6789/0 conn(0x7fd0fc0d0b80 legacy=0x7fd0fc0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd1022b1700 1 -- 172.21.15.79:0/2498984867 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc0ce340 con 0x7fd0fc15c280 2023-06-08T03:51:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd1012af700 1 -- 172.21.15.79:0/2498984867 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774047178 0 0) 0x7fd0fc0cdaa0 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.806 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd1022b1700 1 --2- 172.21.15.79:0/2498984867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc15c280 0x7fd0fc0d23f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009820 tx=0x7fd0ec009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cd9cbecea77271 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd1012af700 1 -- 172.21.15.79:0/2498984867 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec005e30 con 0x7fd0fc15c280 2023-06-08T03:51:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd1012af700 1 -- 172.21.15.79:0/2498984867 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ec004030 con 0x7fd0fc15c280 2023-06-08T03:51:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd1012af700 1 -- 172.21.15.79:0/2498984867 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0043c0 con 0x7fd0fc15c280 2023-06-08T03:51:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 -- 172.21.15.79:0/2498984867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc15c280 msgr2=0x7fd0fc0d23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 --2- 172.21.15.79:0/2498984867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc15c280 0x7fd0fc0d23f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009820 tx=0x7fd0ec009b30 comp rx=0 tx=0).stop 2023-06-08T03:51:44.807 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 -- 172.21.15.79:0/2498984867 shutdown_connections 2023-06-08T03:51:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 --2- 172.21.15.79:0/2498984867 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc15c280 0x7fd0fc0d23f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 -- 172.21.15.79:0/2498984867 >> 172.21.15.79:0/2498984867 conn(0x7fd0fc09bc40 msgr2=0x7fd0fc0d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 -- 172.21.15.79:0/2498984867 shutdown_connections 2023-06-08T03:51:44.808 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.801+0000 7fd104515700 1 -- 172.21.15.79:0/2498984867 wait complete. 2023-06-08T03:51:44.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 Processor -- start 2023-06-08T03:51:44.810 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 -- start start 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0fc0ceb40 0x7fd0fc155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 0x7fd0fc0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0fc15c280 0x7fd0fc06c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0ca860 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0cab40 con 0x7fd0fc0ceb40 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd104515700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0cae20 con 0x7fd0fc15c280 2023-06-08T03:51:44.811 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd101ab0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 0x7fd0fc0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd101ab0700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 0x7fd0fc0afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37388/0 (socket says 172.21.15.79:37388) 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.803+0000 7fd101ab0700 1 -- 172.21.15.79:0/3222003086 learned_addr learned my addr 172.21.15.79:0/3222003086 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd1022b1700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0fc0ceb40 0x7fd0fc155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd102ab2700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0fc15c280 0x7fd0fc06c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd101ab0700 1 -- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0fc15c280 msgr2=0x7fd0fc06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd101ab0700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0fc15c280 0x7fd0fc06c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.812 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd101ab0700 1 -- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0fc0ceb40 msgr2=0x7fd0fc155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd101ab0700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0fc0ceb40 0x7fd0fc155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd101ab0700 1 -- 172.21.15.79:0/3222003086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec009430 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd1022b1700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0fc0ceb40 0x7fd0fc155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd101ab0700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 0x7fd0fc0afa60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b4a0 tx=0x7fd0f800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8011830 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8011990 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.813 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc0cb100 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8011d40 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.804+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc0affa0 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.805+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc0cb620 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.805+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0f801a400 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.806+0000 7fd0f2ffd700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd0e006ace0 0x7fd0e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.806+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd0f8089010 con 0x7fd0fc0d0b80 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.806+0000 7fd1022b1700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd0e006ace0 0x7fd0e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:44.814 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.806+0000 7fd1022b1700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd0e006ace0 0x7fd0e006d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec014040 tx=0x7fd0ec008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:44.815 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:44.810+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0fc0cb620 con 0x7fd0fc0d0b80 2023-06-08T03:51:45.072 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.069+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fd0fc0cb620 con 0x7fd0fc0d0b80 2023-06-08T03:51:45.075 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.072+0000 7fd0f2ffd700 1 -- 172.21.15.79:0/3222003086 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd0fc0cb620 con 0x7fd0fc0d0b80 2023-06-08T03:51:45.077 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd0e006ace0 msgr2=0x7fd0e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd0e006ace0 0x7fd0e006d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec014040 tx=0x7fd0ec008070 comp rx=0 tx=0).stop 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 msgr2=0x7fd0fc0afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 0x7fd0fc0afa60 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b4a0 tx=0x7fd0f800b860 comp rx=0 tx=0).stop 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 shutdown_connections 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd0e006ace0 0x7fd0e006d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd0fc15c280 0x7fd0fc06c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.078 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd0fc0ceb40 0x7fd0fc155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 --2- 172.21.15.79:0/3222003086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0fc0d0b80 0x7fd0fc0afa60 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.074+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 >> 172.21.15.79:0/3222003086 conn(0x7fd0fc09bc40 msgr2=0x7fd0fc0d2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.075+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 shutdown_connections 2023-06-08T03:51:45.079 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.075+0000 7fd104515700 1 -- 172.21.15.79:0/3222003086 wait complete. 2023-06-08T03:51:45.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-06-08T03:51:45.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.207+0000 7fc06e2e5700 1 Processor -- start 2023-06-08T03:51:45.210 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.207+0000 7fc06e2e5700 1 -- start start 2023-06-08T03:51:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06e2e5700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc0680cce80 0x7fc068070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.212 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06e2e5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc06815fb10 0x7fc0680cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06e2e5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc0680b20f0 0x7fc0680b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06e2e5700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc068070bd0 con 0x7fc0680cce80 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06e2e5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680cad50 con 0x7fc0680b20f0 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06e2e5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680ce570 con 0x7fc06815fb10 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc0680cce80 0x7fc068070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc0680cce80 0x7fc068070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37394/0 (socket says 172.21.15.79:37394) 2023-06-08T03:51:45.213 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 -- 172.21.15.79:0/4257224065 learned_addr learned my addr 172.21.15.79:0/4257224065 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc0677fe700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc06815fb10 0x7fc0680cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc06c882700 1 --2- 172.21.15.79:0/4257224065 >> v2:172.21.15.156:3300/0 conn(0x7fc0680b20f0 0x7fc0680b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 -- 172.21.15.79:0/4257224065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc06815fb10 msgr2=0x7fc0680cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc06815fb10 0x7fc0680cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 -- 172.21.15.79:0/4257224065 >> v2:172.21.15.156:3300/0 conn(0x7fc0680b20f0 msgr2=0x7fc0680b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 --2- 172.21.15.79:0/4257224065 >> v2:172.21.15.156:3300/0 conn(0x7fc0680b20f0 0x7fc0680b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 -- 172.21.15.79:0/4257224065 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680cd790 con 0x7fc0680cce80 2023-06-08T03:51:45.214 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc0677fe700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc06815fb10 0x7fc0680cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.208+0000 7fc067fff700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc068070690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc0680cc740 tx=0x7fc05c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85db0780c629417d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc066ffd700 1 -- 172.21.15.79:0/4257224065 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00b870 con 0x7fc0680cce80 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc066ffd700 1 -- 172.21.15.79:0/4257224065 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c00b9d0 con 0x7fc0680cce80 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc066ffd700 1 -- 172.21.15.79:0/4257224065 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c00bd60 con 0x7fc0680cce80 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 -- 172.21.15.79:0/4257224065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 msgr2=0x7fc068070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc068070690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc0680cc740 tx=0x7fc05c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.215 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 -- 172.21.15.79:0/4257224065 shutdown_connections 2023-06-08T03:51:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/4257224065 >> v2:172.21.15.156:3300/0 conn(0x7fc0680b20f0 0x7fc0680b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc068070690 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/4257224065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc06815fb10 0x7fc0680cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 -- 172.21.15.79:0/4257224065 >> 172.21.15.79:0/4257224065 conn(0x7fc06806e1e0 msgr2=0x7fc06806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 -- 172.21.15.79:0/4257224065 shutdown_connections 2023-06-08T03:51:45.216 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.209+0000 7fc06e2e5700 1 -- 172.21.15.79:0/4257224065 wait complete. 2023-06-08T03:51:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.210+0000 7fc06e2e5700 1 Processor -- start 2023-06-08T03:51:45.218 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 -- start start 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0680b20f0 0x7fc0680c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc06815c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc06815fb10 0x7fc06815c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680c42a0 con 0x7fc0680cce80 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06815a630 con 0x7fc06815fb10 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06e2e5700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06815a910 con 0x7fc0680b20f0 2023-06-08T03:51:45.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc0677fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc06815c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc0677fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc06815c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37400/0 (socket says 172.21.15.79:37400) 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc0677fe700 1 -- 172.21.15.79:0/1916666164 learned_addr learned my addr 172.21.15.79:0/1916666164 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc067fff700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0680b20f0 0x7fc0680c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.211+0000 7fc06c882700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc06815fb10 0x7fc06815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc0677fe700 1 -- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0680b20f0 msgr2=0x7fc0680c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc0677fe700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0680b20f0 0x7fc0680c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc0677fe700 1 -- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc06815fb10 msgr2=0x7fc06815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc0677fe700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc06815fb10 0x7fc06815c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc0677fe700 1 -- 172.21.15.79:0/1916666164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c009550 con 0x7fc0680cce80 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc06c882700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc06815fb10 0x7fc06815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc0677fe700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc06815c230 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c750 tx=0x7fc05800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058011800 con 0x7fc0680cce80 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058011960 con 0x7fc0680cce80 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06815abf0 con 0x7fc0680cce80 2023-06-08T03:51:45.221 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058011cd0 con 0x7fc0680cce80 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.212+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680c6720 con 0x7fc0680cce80 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.213+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc068073e90 con 0x7fc0680cce80 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.214+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc058021350 con 0x7fc0680cce80 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.214+0000 7fc064ff9700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc05406ace0 0x7fc05406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.214+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc058089170 con 0x7fc0680cce80 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.214+0000 7fc067fff700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc05406ace0 0x7fc05406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.215+0000 7fc067fff700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc05406ace0 0x7fc05406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc05c014040 tx=0x7fc05c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:45.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.218+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc058057ec0 con 0x7fc0680cce80 2023-06-08T03:51:45.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.478+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc068070120 con 0x7fc0680cce80 2023-06-08T03:51:45.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.481+0000 7fc064ff9700 1 -- 172.21.15.79:0/1916666164 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc058057a50 con 0x7fc0680cce80 2023-06-08T03:51:45.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc05406ace0 msgr2=0x7fc05406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc05406ace0 0x7fc05406d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc05c014040 tx=0x7fc05c008030 comp rx=0 tx=0).stop 2023-06-08T03:51:45.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 msgr2=0x7fc06815c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc06815c230 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c750 tx=0x7fc05800ca60 comp rx=0 tx=0).stop 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 shutdown_connections 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc06815fb10 0x7fc06815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fc05406ace0 0x7fc05406d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0680b20f0 0x7fc0680c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 --2- 172.21.15.79:0/1916666164 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0680cce80 0x7fc06815c230 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.487 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 >> 172.21.15.79:0/1916666164 conn(0x7fc06806e1e0 msgr2=0x7fc0680d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:45.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 shutdown_connections 2023-06-08T03:51:45.488 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.483+0000 7fc06e2e5700 1 -- 172.21.15.79:0/1916666164 wait complete. 2023-06-08T03:51:45.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-06-08T03:51:45.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.616+0000 7f3b6b791700 1 Processor -- start 2023-06-08T03:51:45.619 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.616+0000 7f3b6b791700 1 -- start start 2023-06-08T03:51:45.620 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b6b791700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3b640d0060 0x7f3b640af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b6b791700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3b640cd080 0x7f3b640cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b6b791700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640cae50 con 0x7f3b640b20f0 2023-06-08T03:51:45.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b6b791700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b640cbea0 con 0x7f3b640cd080 2023-06-08T03:51:45.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b6b791700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b640cda30 con 0x7f3b640d0060 2023-06-08T03:51:45.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3b640cd080 0x7f3b640cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.621 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3b640cd080 0x7f3b640cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37408/0 (socket says 172.21.15.79:37408) 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b69d2e700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3b640b20f0 0x7f3b640b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60886/0 (socket says 172.21.15.79:60886) 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 -- 172.21.15.79:0/2977539912 learned_addr learned my addr 172.21.15.79:0/2977539912 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b6952d700 1 --2- 172.21.15.79:0/2977539912 >> v2:172.21.15.156:3300/0 conn(0x7f3b640d0060 0x7f3b640af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 -- 172.21.15.79:0/2977539912 >> v2:172.21.15.156:3300/0 conn(0x7f3b640d0060 msgr2=0x7f3b640af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 --2- 172.21.15.79:0/2977539912 >> v2:172.21.15.156:3300/0 conn(0x7f3b640d0060 0x7f3b640af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 -- 172.21.15.79:0/2977539912 >> v1:172.21.15.79:6789/0 conn(0x7f3b640b20f0 legacy=0x7f3b640b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b5bfff700 1 -- 172.21.15.79:0/2977539912 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671263302 0 0) 0x7f3b640cae50 con 0x7f3b640b20f0 2023-06-08T03:51:45.622 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.617+0000 7f3b68d2c700 1 -- 172.21.15.79:0/2977539912 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6415d930 con 0x7f3b640cd080 2023-06-08T03:51:45.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b68d2c700 1 --2- 172.21.15.79:0/2977539912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640cd080 0x7f3b640cd4f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3b6415d890 tx=0x7f3b60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fda818f258c20e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:45.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b5bfff700 1 -- 172.21.15.79:0/2977539912 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000b870 con 0x7f3b640cd080 2023-06-08T03:51:45.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b5bfff700 1 -- 172.21.15.79:0/2977539912 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6000b9d0 con 0x7f3b640cd080 2023-06-08T03:51:45.623 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b5bfff700 1 -- 172.21.15.79:0/2977539912 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000bd60 con 0x7f3b640cd080 2023-06-08T03:51:45.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 -- 172.21.15.79:0/2977539912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640cd080 msgr2=0x7f3b640cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.625 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2977539912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640cd080 0x7f3b640cd4f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3b6415d890 tx=0x7f3b60009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 -- 172.21.15.79:0/2977539912 shutdown_connections 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2977539912 >> v2:172.21.15.156:3300/0 conn(0x7f3b640d0060 0x7f3b640af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2977539912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640cd080 0x7f3b640cd4f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 -- 172.21.15.79:0/2977539912 >> 172.21.15.79:0/2977539912 conn(0x7f3b6406e1e0 msgr2=0x7f3b6406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 -- 172.21.15.79:0/2977539912 shutdown_connections 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.618+0000 7f3b6b791700 1 -- 172.21.15.79:0/2977539912 wait complete. 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.619+0000 7f3b6b791700 1 Processor -- start 2023-06-08T03:51:45.626 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 -- start start 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 0x7f3b641e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b640cd080 0x7f3b640c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b640d0060 0x7f3b640c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640c8ae0 con 0x7f3b640b20f0 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6414b190 con 0x7f3b640d0060 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6b791700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640c81b0 con 0x7f3b640cd080 2023-06-08T03:51:45.627 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6952d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 0x7f3b641e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6952d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 0x7f3b641e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37412/0 (socket says 172.21.15.79:37412) 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b6952d700 1 -- 172.21.15.79:0/2670766011 learned_addr learned my addr 172.21.15.79:0/2670766011 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b69d2e700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b640d0060 0x7f3b640c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.620+0000 7f3b68d2c700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b640cd080 0x7f3b640c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6952d700 1 -- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b640cd080 msgr2=0x7f3b640c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6952d700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b640cd080 0x7f3b640c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.628 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6952d700 1 -- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b640d0060 msgr2=0x7f3b640c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6952d700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b640d0060 0x7f3b640c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6952d700 1 -- 172.21.15.79:0/2670766011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b60009550 con 0x7f3b640b20f0 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6952d700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 0x7f3b641e5ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400e840 tx=0x7f3b5400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54013610 con 0x7f3b640b20f0 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b54013770 con 0x7f3b640b20f0 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b640c8490 con 0x7f3b640b20f0 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54013b20 con 0x7f3b640b20f0 2023-06-08T03:51:45.629 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.621+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b641e60e0 con 0x7f3b640b20f0 2023-06-08T03:51:45.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.622+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b641e62d0 con 0x7f3b640b20f0 2023-06-08T03:51:45.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.623+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b54013c80 con 0x7f3b640b20f0 2023-06-08T03:51:45.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.623+0000 7f3b59ffb700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3b4806ace0 0x7f3b4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:45.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.623+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3b54089010 con 0x7f3b640b20f0 2023-06-08T03:51:45.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.624+0000 7f3b68d2c700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3b4806ace0 0x7f3b4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:45.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.624+0000 7f3b68d2c700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3b4806ace0 0x7f3b4806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b6415d890 tx=0x7f3b60008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:45.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.627+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b54057cd0 con 0x7f3b640b20f0 2023-06-08T03:51:45.894 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.891+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f3b64073e90 con 0x7f3b640b20f0 2023-06-08T03:51:45.897 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.894+0000 7f3b59ffb700 1 -- 172.21.15.79:0/2670766011 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3b540210a0 con 0x7f3b640b20f0 2023-06-08T03:51:45.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3b4806ace0 msgr2=0x7f3b4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3b4806ace0 0x7f3b4806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3b6415d890 tx=0x7f3b60008030 comp rx=0 tx=0).stop 2023-06-08T03:51:45.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 msgr2=0x7f3b641e5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:45.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 0x7f3b641e5ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400e840 tx=0x7f3b5400ec00 comp rx=0 tx=0).stop 2023-06-08T03:51:45.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 shutdown_connections 2023-06-08T03:51:45.900 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b640d0060 0x7f3b640c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b640b20f0 0x7f3b641e5ba0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f3b4806ace0 0x7f3b4806d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 --2- 172.21.15.79:0/2670766011 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b640cd080 0x7f3b640c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:45.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 >> 172.21.15.79:0/2670766011 conn(0x7f3b6406e1e0 msgr2=0x7f3b640cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:45.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 shutdown_connections 2023-06-08T03:51:45.901 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:45.897+0000 7f3b6b791700 1 -- 172.21.15.79:0/2670766011 wait complete. 2023-06-08T03:51:45.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-06-08T03:51:46.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.029+0000 7fde4c8e9700 1 Processor -- start 2023-06-08T03:51:46.032 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.029+0000 7fde4c8e9700 1 -- start start 2023-06-08T03:51:46.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.030+0000 7fde4c8e9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fde48158d50 0x7fde480ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.030+0000 7fde4c8e9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fde480cc690 0x7fde48071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.030+0000 7fde4c8e9700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde480cd7b0 con 0x7fde4815d3d0 2023-06-08T03:51:46.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.030+0000 7fde4c8e9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde480cda50 con 0x7fde480cc690 2023-06-08T03:51:46.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.030+0000 7fde4c8e9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde480cdd30 con 0x7fde48158d50 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fde480cc690 0x7fde48071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fde480cc690 0x7fde48071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37424/0 (socket says 172.21.15.79:37424) 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 -- 172.21.15.79:0/3609322249 learned_addr learned my addr 172.21.15.79:0/3609322249 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde4659c700 1 --2- 172.21.15.79:0/3609322249 >> v2:172.21.15.156:3300/0 conn(0x7fde48158d50 0x7fde480ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 -- 172.21.15.79:0/3609322249 >> v2:172.21.15.156:3300/0 conn(0x7fde48158d50 msgr2=0x7fde480ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 --2- 172.21.15.79:0/3609322249 >> v2:172.21.15.156:3300/0 conn(0x7fde48158d50 0x7fde480ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 -- 172.21.15.79:0/3609322249 >> v1:172.21.15.156:6789/0 conn(0x7fde4815d3d0 legacy=0x7fde480cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 -- 172.21.15.79:0/3609322249 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480ce010 con 0x7fde480cc690 2023-06-08T03:51:46.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde46d9d700 1 --2- 172.21.15.79:0/3609322249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde48071220 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde4806e5d0 tx=0x7fde4000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae923ebccc6399e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde4559a700 1 -- 172.21.15.79:0/3609322249 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40010070 con 0x7fde480cc690 2023-06-08T03:51:46.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde4559a700 1 -- 172.21.15.79:0/3609322249 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4000dd90 con 0x7fde480cc690 2023-06-08T03:51:46.036 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.031+0000 7fde4559a700 1 -- 172.21.15.79:0/3609322249 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000b310 con 0x7fde480cc690 2023-06-08T03:51:46.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3609322249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 msgr2=0x7fde48071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3609322249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde48071220 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde4806e5d0 tx=0x7fde4000d430 comp rx=0 tx=0).stop 2023-06-08T03:51:46.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3609322249 shutdown_connections 2023-06-08T03:51:46.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3609322249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde48071220 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3609322249 >> v2:172.21.15.156:3300/0 conn(0x7fde48158d50 0x7fde480ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.039 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3609322249 >> 172.21.15.79:0/3609322249 conn(0x7fde480b1d30 msgr2=0x7fde4806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3609322249 shutdown_connections 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.032+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3609322249 wait complete. 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.033+0000 7fde4c8e9700 1 Processor -- start 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.033+0000 7fde4c8e9700 1 -- start start 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4c8e9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde480af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4c8e9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde48158d50 0x7fde480b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.040 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4c8e9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4815d3d0 0x7fde480c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4c8e9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c4cd0 con 0x7fde480cc690 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4c8e9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde48152e30 con 0x7fde4815d3d0 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4c8e9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde480c40c0 con 0x7fde48158d50 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde480af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde480af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37426/0 (socket says 172.21.15.79:37426) 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 -- 172.21.15.79:0/3279781162 learned_addr learned my addr 172.21.15.79:0/3279781162 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:46.041 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde46d9d700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4815d3d0 0x7fde480c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde45d9b700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde48158d50 0x7fde480b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 -- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde48158d50 msgr2=0x7fde480b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde48158d50 0x7fde480b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 -- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4815d3d0 msgr2=0x7fde480c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4815d3d0 0x7fde480c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 -- 172.21.15.79:0/3279781162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4000bd40 con 0x7fde480cc690 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.034+0000 7fde4659c700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde480af040 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00e810 tx=0x7fde3c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.042 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.035+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00c070 con 0x7fde480cc690 2023-06-08T03:51:46.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.035+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c013610 con 0x7fde480cc690 2023-06-08T03:51:46.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.035+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde480c43a0 con 0x7fde480cc690 2023-06-08T03:51:46.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.035+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0139c0 con 0x7fde480cc690 2023-06-08T03:51:46.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.035+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde480c4960 con 0x7fde480cc690 2023-06-08T03:51:46.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.035+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde480c4fb0 con 0x7fde480cc690 2023-06-08T03:51:46.043 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.036+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde3c013b20 con 0x7fde480cc690 2023-06-08T03:51:46.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.036+0000 7fde36ffd700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fde2806ad10 0x7fde2806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.036+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fde3c088f90 con 0x7fde480cc690 2023-06-08T03:51:46.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.037+0000 7fde45d9b700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fde2806ad10 0x7fde2806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.044 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.037+0000 7fde45d9b700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fde2806ad10 0x7fde2806d1c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde38007fd0 tx=0x7fde38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.045 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.041+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fde3c057cf0 con 0x7fde480cc690 2023-06-08T03:51:46.307 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.305+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fde480ce2c0 con 0x7fde480cc690 2023-06-08T03:51:46.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.308+0000 7fde36ffd700 1 -- 172.21.15.79:0/3279781162 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fde3c057880 con 0x7fde480cc690 2023-06-08T03:51:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fde2806ad10 msgr2=0x7fde2806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fde2806ad10 0x7fde2806d1c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fde38007fd0 tx=0x7fde38009040 comp rx=0 tx=0).stop 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 msgr2=0x7fde480af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde480af040 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00e810 tx=0x7fde3c00eb20 comp rx=0 tx=0).stop 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 shutdown_connections 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde4815d3d0 0x7fde480c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fde480cc690 0x7fde480af040 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fde2806ad10 0x7fde2806d1c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 --2- 172.21.15.79:0/3279781162 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fde48158d50 0x7fde480b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 >> 172.21.15.79:0/3279781162 conn(0x7fde480b1d30 msgr2=0x7fde48156670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 shutdown_connections 2023-06-08T03:51:46.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.310+0000 7fde4c8e9700 1 -- 172.21.15.79:0/3279781162 wait complete. 2023-06-08T03:51:46.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:46.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.441+0000 7f7fafdd9700 1 Processor -- start 2023-06-08T03:51:46.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.442+0000 7f7fafdd9700 1 -- start start 2023-06-08T03:51:46.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.442+0000 7f7fafdd9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 0x7f7fa806f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.442+0000 7f7fafdd9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7fa80b0fe0 0x7f7fa80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.442+0000 7f7fafdd9700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa806edd0 con 0x7f7fa815c680 2023-06-08T03:51:46.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fafdd9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa80b19a0 con 0x7f7fa80b0fe0 2023-06-08T03:51:46.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fafdd9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa806f7a0 con 0x7f7fa806bb30 2023-06-08T03:51:46.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fadb75700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7fa815c680 0x7f7fa80ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60910/0 (socket says 172.21.15.79:60910) 2023-06-08T03:51:46.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fadb75700 1 -- 172.21.15.79:0/176857606 learned_addr learned my addr 172.21.15.79:0/176857606 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:46.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fad374700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 0x7f7fa806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fae376700 1 --2- 172.21.15.79:0/176857606 >> v2:172.21.15.156:3300/0 conn(0x7f7fa80b0fe0 0x7f7fa80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76052245 0 0) 0x7f7fa806edd0 con 0x7f7fa815c680 2023-06-08T03:51:46.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f940035b0 con 0x7f7fa815c680 2023-06-08T03:51:46.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3513072568 0 0) 0x7f7f940035b0 con 0x7f7fa815c680 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 msgr2=0x7f7fa806f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 0x7f7fa806f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 >> v2:172.21.15.156:3300/0 conn(0x7f7fa80b0fe0 msgr2=0x7f7fa80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 --2- 172.21.15.79:0/176857606 >> v2:172.21.15.156:3300/0 conn(0x7f7fa80b0fe0 0x7f7fa80b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa806eaa0 con 0x7f7fa815c680 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7f7f98001670 con 0x7f7fa815c680 2023-06-08T03:51:46.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fae376700 1 --2- 172.21.15.79:0/176857606 >> v2:172.21.15.156:3300/0 conn(0x7f7fa80b0fe0 0x7f7fa80b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 >> v1:172.21.15.79:6789/0 conn(0x7f7fa815c680 legacy=0x7f7fa80ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 0x7f7f94003a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f94003ff0 0x7f7f94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f94004d90 0x7f7f94005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80b19a0 con 0x7f7f94004d90 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa806f7a0 con 0x7f7f94003ff0 2023-06-08T03:51:46.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9400bb70 con 0x7f7fa806bb30 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fae376700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f94004d90 0x7f7f94005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fadb75700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 0x7f7f94003a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.443+0000 7f7fad374700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f94003ff0 0x7f7f94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 -- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 msgr2=0x7f7f94003a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa806bb30 0x7f7f94003a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 -- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f94003ff0 msgr2=0x7f7f94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:46.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7f94003ff0 0x7f7f94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 -- 172.21.15.79:0/176857606 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940035b0 con 0x7f7f94004d90 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f94004d90 0x7f7f94005200 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e320 tx=0x7f7f94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adeaede0fda5a5a0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa000edc0 con 0x7f7f94004d90 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa000f040 con 0x7f7f94004d90 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7facb73700 1 -- 172.21.15.79:0/176857606 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00146a0 con 0x7f7f94004d90 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fafdd9700 1 -- 172.21.15.79:0/176857606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f94004d90 msgr2=0x7f7f94005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/176857606 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f94004d90 0x7f7f94005200 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e320 tx=0x7f7f94005880 comp rx=0 tx=0).stop 2023-06-08T03:51:46.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fae376700 1 -- 172.21.15.79:0/176857606 reap_dead start 2023-06-08T03:51:46.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fafdd9700 1 -- 172.21.15.79:0/176857606 shutdown_connections 2023-06-08T03:51:46.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.444+0000 7f7fafdd9700 1 -- 172.21.15.79:0/176857606 >> 172.21.15.79:0/176857606 conn(0x7f7fa80cbfe0 msgr2=0x7f7fa80cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:46.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.445+0000 7f7fafdd9700 1 -- 172.21.15.79:0/176857606 shutdown_connections 2023-06-08T03:51:46.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.445+0000 7f7fafdd9700 1 -- 172.21.15.79:0/176857606 wait complete. 2023-06-08T03:51:46.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 Processor -- start 2023-06-08T03:51:46.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 -- start start 2023-06-08T03:51:46.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f94004d90 0x7f7fa8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 0x7f7fa81f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa80c5b30 0x7f7fa81f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81f2a60 con 0x7f7fa80c56c0 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80b5310 con 0x7f7fa80c5b30 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.446+0000 7f7fafdd9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa8159fe0 con 0x7f7f94004d90 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 0x7f7fa81f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 0x7f7fa81f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37448/0 (socket says 172.21.15.79:37448) 2023-06-08T03:51:46.457 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 -- 172.21.15.79:0/1307102270 learned_addr learned my addr 172.21.15.79:0/1307102270 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fae376700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa80c5b30 0x7f7fa81f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fadb75700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f94004d90 0x7f7fa8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 -- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f94004d90 msgr2=0x7f7fa8159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f94004d90 0x7f7fa8159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 -- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa80c5b30 msgr2=0x7f7fa81f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa80c5b30 0x7f7fa81f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.458 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 -- 172.21.15.79:0/1307102270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80b4790 con 0x7f7fa80c56c0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fadb75700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f94004d90 0x7f7fa8159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fad374700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 0x7f7fa81f1fe0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40119a0 tx=0x7f7fa4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400f940 con 0x7f7fa80c56c0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa400faa0 con 0x7f7fa80c56c0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa80b4a70 con 0x7f7fa80c56c0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400fe30 con 0x7f7fa80c56c0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.447+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa80b5030 con 0x7f7fa80c56c0 2023-06-08T03:51:46.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.448+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa8073e90 con 0x7f7fa80c56c0 2023-06-08T03:51:46.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.449+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fa4014440 con 0x7f7fa80c56c0 2023-06-08T03:51:46.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.449+0000 7f7f9e7fc700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7f8c06ace0 0x7f7f8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.449+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f7fa4089850 con 0x7f7fa80c56c0 2023-06-08T03:51:46.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.449+0000 7f7fadb75700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7f8c06ace0 0x7f7f8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.450+0000 7f7fadb75700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7f8c06ace0 0x7f7f8c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f980010e0 tx=0x7f7f98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.453+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7fa4054ba0 con 0x7f7fa80c56c0 2023-06-08T03:51:46.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.639+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fa80b54e0 con 0x7f7fa80c56c0 2023-06-08T03:51:46.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.640+0000 7f7f9e7fc700 1 -- 172.21.15.79:0/1307102270 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+14762 (secure 0 0 0) 0x7f7fa40581c0 con 0x7f7fa80c56c0 2023-06-08T03:51:46.643 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:46.643 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":34,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:51:43.396808+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:46.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7f8c06ace0 msgr2=0x7f7f8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7f8c06ace0 0x7f7f8c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f980010e0 tx=0x7f7f98009040 comp rx=0 tx=0).stop 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 msgr2=0x7f7fa81f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 0x7f7fa81f1fe0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40119a0 tx=0x7f7fa4011cb0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 shutdown_connections 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa80c5b30 0x7f7fa81f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7f8c06ace0 0x7f7f8c06d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7f94004d90 0x7f7fa8159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 --2- 172.21.15.79:0/1307102270 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7fa80c56c0 0x7f7fa81f1fe0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 >> 172.21.15.79:0/1307102270 conn(0x7f7fa80cbfe0 msgr2=0x7f7fa80b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:46.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 shutdown_connections 2023-06-08T03:51:46.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.642+0000 7f7fafdd9700 1 -- 172.21.15.79:0/1307102270 wait complete. 2023-06-08T03:51:46.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-08T03:50:17.429601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-08T03:50:17.909779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-08T03:51:46.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-08T03:51:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.777+0000 7f6e623ee700 1 Processor -- start 2023-06-08T03:51:46.780 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.777+0000 7f6e623ee700 1 -- start start 2023-06-08T03:51:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e623ee700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6e5c158d50 0x7f6e5c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e623ee700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c15d3d0 0x7f6e5c0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e623ee700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5c0cc920 con 0x7f6e5c0cc4c0 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e623ee700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c0cd7b0 con 0x7f6e5c158d50 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e623ee700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0cda50 con 0x7f6e5c15d3d0 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6e5c158d50 0x7f6e5c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e6098b700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6e5c0cc4c0 0x7f6e5c06eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60934/0 (socket says 172.21.15.79:60934) 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6e5c158d50 0x7f6e5c0ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37454/0 (socket says 172.21.15.79:37454) 2023-06-08T03:51:46.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e6098b700 1 -- 172.21.15.79:0/2414044760 learned_addr learned my addr 172.21.15.79:0/2414044760 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5b7fe700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c15d3d0 0x7f6e5c0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 -- 172.21.15.79:0/2414044760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c15d3d0 msgr2=0x7f6e5c0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c15d3d0 0x7f6e5c0cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 -- 172.21.15.79:0/2414044760 >> v1:172.21.15.79:6789/0 conn(0x7f6e5c0cc4c0 legacy=0x7f6e5c06eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 -- 172.21.15.79:0/2414044760 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0cdd30 con 0x7f6e5c158d50 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5affd700 1 -- 172.21.15.79:0/2414044760 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774738382 0 0) 0x7f6e5c0cc920 con 0x7f6e5c0cc4c0 2023-06-08T03:51:46.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5b7fe700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c15d3d0 0x7f6e5c0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5bfff700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0ca360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000bae0 tx=0x7f6e5000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adca774fd7972c95 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5affd700 1 -- 172.21.15.79:0/2414044760 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000dd30 con 0x7f6e5c158d50 2023-06-08T03:51:46.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.778+0000 7f6e5affd700 1 -- 172.21.15.79:0/2414044760 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5000de90 con 0x7f6e5c158d50 2023-06-08T03:51:46.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 -- 172.21.15.79:0/2414044760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 msgr2=0x7f6e5c0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0ca360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000bae0 tx=0x7f6e5000bdf0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 -- 172.21.15.79:0/2414044760 shutdown_connections 2023-06-08T03:51:46.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0ca360 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2414044760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c15d3d0 0x7f6e5c0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 -- 172.21.15.79:0/2414044760 >> 172.21.15.79:0/2414044760 conn(0x7f6e5c0b1d30 msgr2=0x7f6e5c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 -- 172.21.15.79:0/2414044760 shutdown_connections 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.779+0000 7f6e623ee700 1 -- 172.21.15.79:0/2414044760 wait complete. 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.780+0000 7f6e623ee700 1 Processor -- start 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.781+0000 7f6e623ee700 1 -- start start 2023-06-08T03:51:46.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c0cc4c0 0x7f6e5c14bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c15d3d0 0x7f6e5c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0c69b0 con 0x7f6e5c158d50 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0c6c90 con 0x7f6e5c15d3d0 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0c6f70 con 0x7f6e5c0cc4c0 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37462/0 (socket says 172.21.15.79:37462) 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 -- 172.21.15.79:0/2350543590 learned_addr learned my addr 172.21.15.79:0/2350543590 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e6098b700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c15d3d0 0x7f6e5c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5bfff700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c0cc4c0 0x7f6e5c14bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 -- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c0cc4c0 msgr2=0x7f6e5c14bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c0cc4c0 0x7f6e5c14bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 -- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c15d3d0 msgr2=0x7f6e5c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c15d3d0 0x7f6e5c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 -- 172.21.15.79:0/2350543590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5000b740 con 0x7f6e5c158d50 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e5b7fe700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0c7a50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c750 tx=0x7f6e4c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c011800 con 0x7f6e5c158d50 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c011960 con 0x7f6e5c158d50 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c0c7250 con 0x7f6e5c158d50 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c011cd0 con 0x7f6e5c158d50 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.782+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c0c6640 con 0x7f6e5c158d50 2023-06-08T03:51:46.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.783+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c0c7770 con 0x7f6e5c158d50 2023-06-08T03:51:46.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.784+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6e4c021350 con 0x7f6e5c158d50 2023-06-08T03:51:46.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.784+0000 7f6e58ff9700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6e4806ace0 0x7f6e4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:46.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.784+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6e4c089270 con 0x7f6e5c158d50 2023-06-08T03:51:46.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.784+0000 7f6e5bfff700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6e4806ace0 0x7f6e4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:46.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.785+0000 7f6e5bfff700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6e4806ace0 0x7f6e4806d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000c00 tx=0x7f6e50013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:46.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.788+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6e5c0c7770 con 0x7f6e5c158d50 2023-06-08T03:51:46.975 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.972+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6e5c0c7770 con 0x7f6e5c158d50 2023-06-08T03:51:46.976 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.973+0000 7f6e58ff9700 1 -- 172.21.15.79:0/2350543590 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6e5c0c7770 con 0x7f6e5c158d50 2023-06-08T03:51:46.976 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 8 2023-06-08T03:51:46.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6e4806ace0 msgr2=0x7f6e4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6e4806ace0 0x7f6e4806d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000c00 tx=0x7f6e50013000 comp rx=0 tx=0).stop 2023-06-08T03:51:46.978 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 msgr2=0x7f6e5c0c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0c7a50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c750 tx=0x7f6e4c00cb10 comp rx=0 tx=0).stop 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 shutdown_connections 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e5c15d3d0 0x7f6e5c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f6e4806ace0 0x7f6e4806d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e5c0cc4c0 0x7f6e5c14bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 --2- 172.21.15.79:0/2350543590 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e5c158d50 0x7f6e5c0c7a50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:46.979 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 >> 172.21.15.79:0/2350543590 conn(0x7f6e5c0b1d30 msgr2=0x7f6e5c06a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:46.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 shutdown_connections 2023-06-08T03:51:46.980 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:46.975+0000 7f6e623ee700 1 -- 172.21.15.79:0/2350543590 wait complete. 2023-06-08T03:51:46.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-08T03:51:47.112 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.109+0000 7fca4098c700 1 Processor -- start 2023-06-08T03:51:47.113 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.110+0000 7fca4098c700 1 -- start start 2023-06-08T03:51:47.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.110+0000 7fca4098c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.110+0000 7fca4098c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fca3c069d50 0x7fca3c06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.114 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.110+0000 7fca4098c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0cd140 con 0x7fca3c071fe0 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.110+0000 7fca4098c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c06a710 con 0x7fca3c069d50 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.110+0000 7fca4098c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c071110 con 0x7fca3c15c1f0 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fca3c069d50 0x7fca3c06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fca3c069d50 0x7fca3c06a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37470/0 (socket says 172.21.15.79:37470) 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 -- 172.21.15.79:0/274738950 learned_addr learned my addr 172.21.15.79:0/274738950 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3a59c700 1 --2- 172.21.15.79:0/274738950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.115 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 -- 172.21.15.79:0/274738950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 msgr2=0x7fca3c06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 --2- 172.21.15.79:0/274738950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 -- 172.21.15.79:0/274738950 >> v1:172.21.15.156:6789/0 conn(0x7fca3c071fe0 legacy=0x7fca3c0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 -- 172.21.15.79:0/274738950 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c071be0 con 0x7fca3c069d50 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3ad9d700 1 --2- 172.21.15.79:0/274738950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c06a1d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fca34009e50 tx=0x7fca3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5607de36e58089 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3959a700 1 -- 172.21.15.79:0/274738950 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400e070 con 0x7fca3c069d50 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3959a700 1 -- 172.21.15.79:0/274738950 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3400b910 con 0x7fca3c069d50 2023-06-08T03:51:47.116 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.111+0000 7fca3959a700 1 -- 172.21.15.79:0/274738950 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3400bcc0 con 0x7fca3c069d50 2023-06-08T03:51:47.118 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 -- 172.21.15.79:0/274738950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 msgr2=0x7fca3c06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 --2- 172.21.15.79:0/274738950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c06a1d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fca34009e50 tx=0x7fca3400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:47.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 -- 172.21.15.79:0/274738950 shutdown_connections 2023-06-08T03:51:47.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 --2- 172.21.15.79:0/274738950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c06a1d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 --2- 172.21.15.79:0/274738950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 -- 172.21.15.79:0/274738950 >> 172.21.15.79:0/274738950 conn(0x7fca3c0b1d30 msgr2=0x7fca3c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:47.119 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 -- 172.21.15.79:0/274738950 shutdown_connections 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.112+0000 7fca4098c700 1 -- 172.21.15.79:0/274738950 wait complete. 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.113+0000 7fca4098c700 1 Processor -- start 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.113+0000 7fca4098c700 1 -- start start 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca4098c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca4098c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c071fe0 0x7fca3c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca4098c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca4098c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c155d40 con 0x7fca3c069d50 2023-06-08T03:51:47.120 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca4098c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c153a20 con 0x7fca3c071fe0 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca4098c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c153d00 con 0x7fca3c15c1f0 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37474/0 (socket says 172.21.15.79:37474) 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 -- 172.21.15.79:0/120792768 learned_addr learned my addr 172.21.15.79:0/120792768 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca39d9b700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c071fe0 0x7fca3c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3ad9d700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.121 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 -- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 msgr2=0x7fca3c1531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c1531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 -- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c071fe0 msgr2=0x7fca3c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c071fe0 0x7fca3c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 -- 172.21.15.79:0/120792768 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009b30 con 0x7fca3c069d50 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.114+0000 7fca3a59c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c155740 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fca3000e870 tx=0x7fca3000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.115+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30009590 con 0x7fca3c069d50 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.115+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca30013680 con 0x7fca3c069d50 2023-06-08T03:51:47.122 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.115+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c0c7810 con 0x7fca3c069d50 2023-06-08T03:51:47.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.115+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300139f0 con 0x7fca3c069d50 2023-06-08T03:51:47.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.115+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c0c7dd0 con 0x7fca3c069d50 2023-06-08T03:51:47.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.116+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c073e90 con 0x7fca3c069d50 2023-06-08T03:51:47.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.116+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca30013b50 con 0x7fca3c069d50 2023-06-08T03:51:47.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.117+0000 7fca2affd700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fca1c06ace0 0x7fca1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.117+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fca30088fe0 con 0x7fca3c069d50 2023-06-08T03:51:47.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.117+0000 7fca39d9b700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fca1c06ace0 0x7fca1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.117+0000 7fca39d9b700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fca1c06ace0 0x7fca1c06d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007fd0 tx=0x7fca2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.121+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fca30057dc0 con 0x7fca3c069d50 2023-06-08T03:51:47.310 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.308+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca3c072ec0 con 0x7fca3c069d50 2023-06-08T03:51:47.311 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.308+0000 7fca2affd700 1 -- 172.21.15.79:0/120792768 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca30057950 con 0x7fca3c069d50 2023-06-08T03:51:47.311 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2023-06-08T03:51:47.313 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fca1c06ace0 msgr2=0x7fca1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fca1c06ace0 0x7fca1c06d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca2c007fd0 tx=0x7fca2c009040 comp rx=0 tx=0).stop 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 msgr2=0x7fca3c155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c155740 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fca3000e870 tx=0x7fca3000eb80 comp rx=0 tx=0).stop 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 shutdown_connections 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fca1c06ace0 0x7fca1c06d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fca3c15c1f0 0x7fca3c1531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.314 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fca3c069d50 0x7fca3c155740 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 --2- 172.21.15.79:0/120792768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca3c071fe0 0x7fca3c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 >> 172.21.15.79:0/120792768 conn(0x7fca3c0b1d30 msgr2=0x7fca3c0b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:47.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 shutdown_connections 2023-06-08T03:51:47.315 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.310+0000 7fca4098c700 1 -- 172.21.15.79:0/120792768 wait complete. 2023-06-08T03:51:47.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-08T03:51:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.441+0000 7f38e6555700 1 Processor -- start 2023-06-08T03:51:47.444 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.441+0000 7f38e6555700 1 -- start start 2023-06-08T03:51:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38e6555700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f38e015f6c0 0x7f38e0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38e6555700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e006f210 0x7f38e006a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38e6555700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e00cee20 con 0x7f38e00cd6c0 2023-06-08T03:51:47.445 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38e6555700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e00cf330 con 0x7f38e015f6c0 2023-06-08T03:51:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38e6555700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e006fa00 con 0x7f38e006f210 2023-06-08T03:51:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38dffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f38e015f6c0 0x7f38e0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38e4af2700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e006f210 0x7f38e006a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38df7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f38e00cd6c0 0x7f38e00cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:34888/0 (socket says 172.21.15.79:34888) 2023-06-08T03:51:47.446 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.442+0000 7f38df7fe700 1 -- 172.21.15.79:0/2227559694 learned_addr learned my addr 172.21.15.79:0/2227559694 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:47.447 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38e4af2700 1 -- 172.21.15.79:0/2227559694 >> v2:172.21.15.156:3300/0 conn(0x7f38e015f6c0 msgr2=0x7f38e0069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38e4af2700 1 --2- 172.21.15.79:0/2227559694 >> v2:172.21.15.156:3300/0 conn(0x7f38e015f6c0 0x7f38e0069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38e4af2700 1 -- 172.21.15.79:0/2227559694 >> v1:172.21.15.156:6789/0 conn(0x7f38e00cd6c0 legacy=0x7f38e00cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38e4af2700 1 -- 172.21.15.79:0/2227559694 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00cd1e0 con 0x7f38e006f210 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38dffff700 1 --2- 172.21.15.79:0/2227559694 >> v2:172.21.15.156:3300/0 conn(0x7f38e015f6c0 0x7f38e0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38e4af2700 1 --2- 172.21.15.79:0/2227559694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e006f210 0x7f38e006a290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f8de9ef5dc5e22 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38deffd700 1 -- 172.21.15.79:0/2227559694 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d800e070 con 0x7f38e006f210 2023-06-08T03:51:47.448 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38deffd700 1 -- 172.21.15.79:0/2227559694 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d800bb20 con 0x7f38e006f210 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.443+0000 7f38deffd700 1 -- 172.21.15.79:0/2227559694 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8012430 con 0x7f38e006f210 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 -- 172.21.15.79:0/2227559694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e006f210 msgr2=0x7f38e006a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 --2- 172.21.15.79:0/2227559694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e006f210 0x7f38e006a290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38d8009d90 tx=0x7f38d800b330 comp rx=0 tx=0).stop 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 -- 172.21.15.79:0/2227559694 shutdown_connections 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 --2- 172.21.15.79:0/2227559694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e006f210 0x7f38e006a290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 --2- 172.21.15.79:0/2227559694 >> v2:172.21.15.156:3300/0 conn(0x7f38e015f6c0 0x7f38e0069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 -- 172.21.15.79:0/2227559694 >> 172.21.15.79:0/2227559694 conn(0x7f38e00b1d20 msgr2=0x7f38e006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:47.449 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 -- 172.21.15.79:0/2227559694 shutdown_connections 2023-06-08T03:51:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.444+0000 7f38e6555700 1 -- 172.21.15.79:0/2227559694 wait complete. 2023-06-08T03:51:47.450 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.445+0000 7f38e6555700 1 Processor -- start 2023-06-08T03:51:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.445+0000 7f38e6555700 1 -- start start 2023-06-08T03:51:47.451 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e6555700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38e006f210 0x7f38e014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e6555700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e00cd6c0 0x7f38e0152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e6555700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 0x7f38e00b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e6555700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0153380 con 0x7f38e015f6c0 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e6555700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e01516c0 con 0x7f38e006f210 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e6555700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e01519a0 con 0x7f38e00cd6c0 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 0x7f38e00b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.452 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 0x7f38e00b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37492/0 (socket says 172.21.15.79:37492) 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 -- 172.21.15.79:0/2204872266 learned_addr learned my addr 172.21.15.79:0/2204872266 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38dffff700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38e006f210 0x7f38e014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38df7fe700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e00cd6c0 0x7f38e0152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 -- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e00cd6c0 msgr2=0x7f38e0152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e00cd6c0 0x7f38e0152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 -- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38e006f210 msgr2=0x7f38e014f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.453 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38e006f210 0x7f38e014f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 -- 172.21.15.79:0/2204872266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40094a0 con 0x7f38e015f6c0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38e4af2700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 0x7f38e00b49a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f38d8014040 tx=0x7f38d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.446+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8012430 con 0x7f38e015f6c0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.447+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8020070 con 0x7f38e015f6c0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.447+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d8009a70 con 0x7f38e015f6c0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.447+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8003ea0 con 0x7f38e015f6c0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.447+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e0151f60 con 0x7f38e015f6c0 2023-06-08T03:51:47.454 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.447+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e0073e90 con 0x7f38e015f6c0 2023-06-08T03:51:47.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.449+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38d8004280 con 0x7f38e015f6c0 2023-06-08T03:51:47.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.449+0000 7f38dcff9700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38cc06ace0 0x7f38cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.455 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.449+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f38d8022070 con 0x7f38e015f6c0 2023-06-08T03:51:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.449+0000 7f38dffff700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38cc06ace0 0x7f38cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.450+0000 7f38dffff700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38cc06ace0 0x7f38cc06d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009c60 tx=0x7f38d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.452+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38d8056980 con 0x7f38e015f6c0 2023-06-08T03:51:47.643 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.640+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f38e00ae8f0 con 0x7f38e015f6c0 2023-06-08T03:51:47.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.641+0000 7f38dcff9700 1 -- 172.21.15.79:0/2204872266 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+126 (secure 0 0 0) 0x7f38d8059fa0 con 0x7f38e015f6c0 2023-06-08T03:51:47.644 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:47.644 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2023-06-08T03:51:47.644 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:47.644 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": "rbd" 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout: { 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 2, 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:47.645 INFO:teuthology.orchestra.run.smithi079.stdout:] 2023-06-08T03:51:47.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38cc06ace0 msgr2=0x7f38cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.646 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38cc06ace0 0x7f38cc06d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009c60 tx=0x7f38d4008040 comp rx=0 tx=0).stop 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 msgr2=0x7f38e00b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 0x7f38e00b49a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f38d8014040 tx=0x7f38d8004ad0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 shutdown_connections 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38e015f6c0 0x7f38e00b49a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38e006f210 0x7f38e014f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f38cc06ace0 0x7f38cc06d190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 --2- 172.21.15.79:0/2204872266 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f38e00cd6c0 0x7f38e0152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 >> 172.21.15.79:0/2204872266 conn(0x7f38e00b1d20 msgr2=0x7f38e0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 shutdown_connections 2023-06-08T03:51:47.648 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.643+0000 7f38e6555700 1 -- 172.21.15.79:0/2204872266 wait complete. 2023-06-08T03:51:47.658 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-08T03:51:47.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-08T03:51:47.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.778+0000 7f8e4adc1700 1 Processor -- start 2023-06-08T03:51:47.781 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.778+0000 7f8e4adc1700 1 -- start start 2023-06-08T03:51:47.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e4adc1700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8e440cf170 0x7f8e440ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.782 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e4adc1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8e440cd060 0x7f8e440cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e4adc1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440cd9e0 con 0x7f8e440cf680 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e4adc1700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440caf40 con 0x7f8e440cf170 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e4adc1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440ce760 con 0x7f8e440cd060 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8e440cf170 0x7f8e440ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8e440cf170 0x7f8e440ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37496/0 (socket says 172.21.15.79:37496) 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 -- 172.21.15.79:0/1564694458 learned_addr learned my addr 172.21.15.79:0/1564694458 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:47.783 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 -- 172.21.15.79:0/1564694458 >> v2:172.21.15.156:3300/0 conn(0x7f8e440cd060 msgr2=0x7f8e440cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 --2- 172.21.15.79:0/1564694458 >> v2:172.21.15.156:3300/0 conn(0x7f8e440cd060 0x7f8e440cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 -- 172.21.15.79:0/1564694458 >> v1:172.21.15.156:6789/0 conn(0x7f8e440cf680 legacy=0x7f8e440af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 -- 172.21.15.79:0/1564694458 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440cc690 con 0x7f8e440cf170 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e48b5d700 1 --2- 172.21.15.79:0/1564694458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e440ce220 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8e44072150 tx=0x7f8e38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfada75de4dae0b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e437fe700 1 -- 172.21.15.79:0/1564694458 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004230 con 0x7f8e440cf170 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e437fe700 1 -- 172.21.15.79:0/1564694458 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38004390 con 0x7f8e440cf170 2023-06-08T03:51:47.784 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.779+0000 7f8e437fe700 1 -- 172.21.15.79:0/1564694458 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380036a0 con 0x7f8e440cf170 2023-06-08T03:51:47.786 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1564694458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 msgr2=0x7f8e440ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1564694458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e440ce220 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8e44072150 tx=0x7f8e38009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1564694458 shutdown_connections 2023-06-08T03:51:47.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1564694458 >> v2:172.21.15.156:3300/0 conn(0x7f8e440cd060 0x7f8e440cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1564694458 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e440ce220 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1564694458 >> 172.21.15.79:0/1564694458 conn(0x7f8e4406e1e0 msgr2=0x7f8e4406e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1564694458 shutdown_connections 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.780+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1564694458 wait complete. 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.781+0000 7f8e4adc1700 1 Processor -- start 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.781+0000 7f8e4adc1700 1 -- start start 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4adc1700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e440cd060 0x7f8e44153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4adc1700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e441f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.788 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4adc1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e440cf680 0x7f8e440b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4adc1700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441535a0 con 0x7f8e440cf170 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4adc1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440c4180 con 0x7f8e440cf680 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4adc1700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441f0d00 con 0x7f8e440cd060 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e441f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e441f07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37500/0 (socket says 172.21.15.79:37500) 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 -- 172.21.15.79:0/1063859915 learned_addr learned my addr 172.21.15.79:0/1063859915 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:47.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4935e700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e440cf680 0x7f8e440b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e48b5d700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e440cd060 0x7f8e44153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 -- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e440cd060 msgr2=0x7f8e44153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e440cd060 0x7f8e44153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 -- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e440cf680 msgr2=0x7f8e440b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e440cf680 0x7f8e440b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e43fff700 1 -- 172.21.15.79:0/1063859915 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38009550 con 0x7f8e440cf170 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.782+0000 7f8e4935e700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e440cf680 0x7f8e440b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:47.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.783+0000 7f8e43fff700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e441f07c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b470 tx=0x7f8e3000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.783+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30011830 con 0x7f8e440cf170 2023-06-08T03:51:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.783+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30011990 con 0x7f8e440cf170 2023-06-08T03:51:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.783+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30011d40 con 0x7f8e440cf170 2023-06-08T03:51:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.783+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440c4390 con 0x7f8e440cf170 2023-06-08T03:51:47.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.783+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e440c4950 con 0x7f8e440cf170 2023-06-08T03:51:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.784+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e44073e90 con 0x7f8e440cf170 2023-06-08T03:51:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.784+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e3001a400 con 0x7f8e440cf170 2023-06-08T03:51:47.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.785+0000 7f8e417fa700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8e2806ace0 0x7f8e2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.785+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8e30089280 con 0x7f8e440cf170 2023-06-08T03:51:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.785+0000 7f8e48b5d700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8e2806ace0 0x7f8e2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.785+0000 7f8e48b5d700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8e2806ace0 0x7f8e2806d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e38008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:47.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.789+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8e300545d0 con 0x7f8e440cf170 2023-06-08T03:51:47.981 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:47.978+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e44070120 con 0x7f8e440cf170 2023-06-08T03:51:48.231 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.228+0000 7f8e417fa700 1 -- 172.21.15.79:0/1063859915 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8e30057bf0 con 0x7f8e440cf170 2023-06-08T03:51:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.231+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8e2806ace0 msgr2=0x7f8e2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.231+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8e2806ace0 0x7f8e2806d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e38008030 comp rx=0 tx=0).stop 2023-06-08T03:51:48.234 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.231+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 msgr2=0x7f8e441f07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:48.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.231+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e441f07c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b470 tx=0x7f8e3000b830 comp rx=0 tx=0).stop 2023-06-08T03:51:48.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.232+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 shutdown_connections 2023-06-08T03:51:48.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.232+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e440cf680 0x7f8e440b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:48.235 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.232+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f8e2806ace0 0x7f8e2806d190 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e38000c00 tx=0x7f8e38008030 comp rx=0 tx=0).stop 2023-06-08T03:51:48.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.232+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e440cd060 0x7f8e44153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:48.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.232+0000 7f8e4adc1700 1 --2- 172.21.15.79:0/1063859915 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8e440cf170 0x7f8e441f07c0 secure :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b470 tx=0x7f8e3000b830 comp rx=0 tx=0).stop 2023-06-08T03:51:48.236 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.232+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 >> 172.21.15.79:0/1063859915 conn(0x7f8e4406e1e0 msgr2=0x7f8e441590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:48.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.234+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 shutdown_connections 2023-06-08T03:51:48.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.234+0000 7f8e4adc1700 1 -- 172.21.15.79:0/1063859915 wait complete. 2023-06-08T03:51:48.238 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2023-06-08T03:51:48.249 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-08T03:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.372+0000 7fb4bcd2b700 1 Processor -- start 2023-06-08T03:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.372+0000 7fb4bcd2b700 1 -- start start 2023-06-08T03:51:48.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4bcd2b700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb4b806a460 0x7fb4b8071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:48.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4bcd2b700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80cd140 con 0x7fb4b815c1f0 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4bcd2b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b8071110 con 0x7fb4b806d8a0 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4bcd2b700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b8071be0 con 0x7fb4b806a460 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b6d9d700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb4b806a460 0x7fb4b8071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b659c700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb4b815c1f0 0x7fb4b80cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60986/0 (socket says 172.21.15.79:60986) 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b659c700 1 -- 172.21.15.79:0/2804912178 learned_addr learned my addr 172.21.15.79:0/2804912178 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/2804912178 >> v1:172.21.15.156:6789/0 conn(0x7fb4b806d8a0 legacy=0x7fb4b8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:48.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/2804912178 >> v1:172.21.15.79:6789/0 conn(0x7fb4b815c1f0 legacy=0x7fb4b80cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:48.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/2804912178 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80cd390 con 0x7fb4b806a460 2023-06-08T03:51:48.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.373+0000 7fb4b559a700 1 -- 172.21.15.79:0/2804912178 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892671846 0 0) 0x7fb4b80cd140 con 0x7fb4b815c1f0 2023-06-08T03:51:48.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4b6d9d700 1 --2- 172.21.15.79:0/2804912178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b806a460 0x7fb4b8071fe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009d90 tx=0x7fb4b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dc9b8e3560415d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:48.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4b559a700 1 -- 172.21.15.79:0/2804912178 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000e070 con 0x7fb4b806a460 2023-06-08T03:51:48.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4b559a700 1 -- 172.21.15.79:0/2804912178 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b000bb90 con 0x7fb4b806a460 2023-06-08T03:51:48.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4b559a700 1 -- 172.21.15.79:0/2804912178 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0003a20 con 0x7fb4b806a460 2023-06-08T03:51:48.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/2804912178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b806a460 msgr2=0x7fb4b8071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:48.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/2804912178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b806a460 0x7fb4b8071fe0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009d90 tx=0x7fb4b000b330 comp rx=0 tx=0).stop 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/2804912178 shutdown_connections 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/2804912178 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b806a460 0x7fb4b8071fe0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/2804912178 >> 172.21.15.79:0/2804912178 conn(0x7fb4b80b1d30 msgr2=0x7fb4b806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/2804912178 shutdown_connections 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.374+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/2804912178 wait complete. 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 Processor -- start 2023-06-08T03:51:48.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 -- start start 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b806a460 0x7fb4b81526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b806d8a0 0x7fb4b815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 0x7fb4b815e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8157730 con 0x7fb4b815c1f0 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8157a10 con 0x7fb4b806d8a0 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.376+0000 7fb4bcd2b700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8157cf0 con 0x7fb4b806a460 2023-06-08T03:51:48.382 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 0x7fb4b815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 0x7fb4b815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37518/0 (socket says 172.21.15.79:37518) 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/1973963813 learned_addr learned my addr 172.21.15.79:0/1973963813 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b806a460 msgr2=0x7fb4b81526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b806a460 0x7fb4b81526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b806d8a0 msgr2=0x7fb4b815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b806d8a0 0x7fb4b815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 -- 172.21.15.79:0/1973963813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0009a70 con 0x7fb4b815c1f0 2023-06-08T03:51:48.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4b6d9d700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 0x7fb4b815e900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0004000 tx=0x7fb4b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:48.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0012630 con 0x7fb4b815c1f0 2023-06-08T03:51:48.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b00127b0 con 0x7fb4b815c1f0 2023-06-08T03:51:48.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b8157fd0 con 0x7fb4b815c1f0 2023-06-08T03:51:48.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0012b60 con 0x7fb4b815c1f0 2023-06-08T03:51:48.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.377+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b81587d0 con 0x7fb4b815c1f0 2023-06-08T03:51:48.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.378+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b81584f0 con 0x7fb4b815c1f0 2023-06-08T03:51:48.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.379+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4b0012cc0 con 0x7fb4b815c1f0 2023-06-08T03:51:48.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.379+0000 7fb4a6ffd700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb49806ad10 0x7fb49806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:48.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.379+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fb4b008b980 con 0x7fb4b815c1f0 2023-06-08T03:51:48.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.379+0000 7fb4b659c700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb49806ad10 0x7fb49806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:48.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.380+0000 7fb4b659c700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb49806ad10 0x7fb49806d1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac001610 tx=0x7fb4ac0079f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:48.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.383+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb4b81584f0 con 0x7fb4b815c1f0 2023-06-08T03:51:48.574 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:48.571+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4b81584f0 con 0x7fb4b815c1f0 2023-06-08T03:51:49.237 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.234+0000 7fb4a6ffd700 1 -- 172.21.15.79:0/1973963813 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb4b81584f0 con 0x7fb4b815c1f0 2023-06-08T03:51:49.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.239+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb49806ad10 msgr2=0x7fb49806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.239+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb49806ad10 0x7fb49806d1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac001610 tx=0x7fb4ac0079f0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.242 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.239+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 msgr2=0x7fb4b815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.243 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.239+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 0x7fb4b815e900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0004000 tx=0x7fb4b0004ad0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.248+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 shutdown_connections 2023-06-08T03:51:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.248+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb4b815c1f0 0x7fb4b815e900 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.251 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.248+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb49806ad10 0x7fb49806d1c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.248+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4b806a460 0x7fb4b81526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.248+0000 7fb4bcd2b700 1 --2- 172.21.15.79:0/1973963813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4b806d8a0 0x7fb4b815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.248+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 >> 172.21.15.79:0/1973963813 conn(0x7fb4b80b1d30 msgr2=0x7fb4b80cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.249+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 shutdown_connections 2023-06-08T03:51:49.252 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.249+0000 7fb4bcd2b700 1 -- 172.21.15.79:0/1973963813 wait complete. 2023-06-08T03:51:49.257 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2023-06-08T03:51:49.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-08T03:51:49.395 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.393+0000 7ff9d4269700 1 Processor -- start 2023-06-08T03:51:49.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.393+0000 7ff9d4269700 1 -- start start 2023-06-08T03:51:49.396 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.393+0000 7ff9d4269700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc15c1f0 0x7ff9cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d4269700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff9cc069d50 0x7ff9cc06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d4269700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc0cd140 con 0x7ff9cc071fe0 2023-06-08T03:51:49.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d4269700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc06a710 con 0x7ff9cc069d50 2023-06-08T03:51:49.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d4269700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc071110 con 0x7ff9cc15c1f0 2023-06-08T03:51:49.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d1804700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff9cc071fe0 0x7ff9cc0cebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:60998/0 (socket says 172.21.15.79:60998) 2023-06-08T03:51:49.397 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d1804700 1 -- 172.21.15.79:0/75174119 learned_addr learned my addr 172.21.15.79:0/75174119 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:49.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2005700 1 --2- 172.21.15.79:0/75174119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc15c1f0 0x7ff9cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.398 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2806700 1 --2- 172.21.15.79:0/75174119 >> v2:172.21.15.156:3300/0 conn(0x7ff9cc069d50 0x7ff9cc06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d1003700 1 -- 172.21.15.79:0/75174119 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417786411 0 0) 0x7ff9cc0cd140 con 0x7ff9cc071fe0 2023-06-08T03:51:49.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d1003700 1 -- 172.21.15.79:0/75174119 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b80035b0 con 0x7ff9cc071fe0 2023-06-08T03:51:49.399 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2005700 1 -- 172.21.15.79:0/75174119 >> v2:172.21.15.156:3300/0 conn(0x7ff9cc069d50 msgr2=0x7ff9cc06a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2005700 1 --2- 172.21.15.79:0/75174119 >> v2:172.21.15.156:3300/0 conn(0x7ff9cc069d50 0x7ff9cc06a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2005700 1 -- 172.21.15.79:0/75174119 >> v1:172.21.15.79:6789/0 conn(0x7ff9cc071fe0 legacy=0x7ff9cc0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2005700 1 -- 172.21.15.79:0/75174119 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc071be0 con 0x7ff9cc15c1f0 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2806700 1 --2- 172.21.15.79:0/75174119 >> v2:172.21.15.156:3300/0 conn(0x7ff9cc069d50 0x7ff9cc06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.394+0000 7ff9d2005700 1 --2- 172.21.15.79:0/75174119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc15c1f0 0x7ff9cc06d8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009840 tx=0x7ff9c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742c1ffea9223a79 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d1003700 1 -- 172.21.15.79:0/75174119 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0011830 con 0x7ff9cc15c1f0 2023-06-08T03:51:49.400 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d1003700 1 -- 172.21.15.79:0/75174119 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c0011990 con 0x7ff9cc15c1f0 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d1003700 1 -- 172.21.15.79:0/75174119 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0011d20 con 0x7ff9cc15c1f0 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 -- 172.21.15.79:0/75174119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc15c1f0 msgr2=0x7ff9cc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 --2- 172.21.15.79:0/75174119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc15c1f0 0x7ff9cc06d8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009840 tx=0x7ff9c0009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 -- 172.21.15.79:0/75174119 shutdown_connections 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 --2- 172.21.15.79:0/75174119 >> v2:172.21.15.156:3300/0 conn(0x7ff9cc069d50 0x7ff9cc06a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 --2- 172.21.15.79:0/75174119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc15c1f0 0x7ff9cc06d8a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 -- 172.21.15.79:0/75174119 >> 172.21.15.79:0/75174119 conn(0x7ff9cc0b1d30 msgr2=0x7ff9cc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:49.401 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 -- 172.21.15.79:0/75174119 shutdown_connections 2023-06-08T03:51:49.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.395+0000 7ff9d4269700 1 -- 172.21.15.79:0/75174119 wait complete. 2023-06-08T03:51:49.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.396+0000 7ff9d4269700 1 Processor -- start 2023-06-08T03:51:49.402 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 -- start start 2023-06-08T03:51:49.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc069d50 0x7ff9cc1f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 0x7ff9cc146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9cc15c1f0 0x7ff9cc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1f5e20 con 0x7ff9cc071fe0 2023-06-08T03:51:49.404 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc15d610 con 0x7ff9cc15c1f0 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.397+0000 7ff9d4269700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc15d8b0 con 0x7ff9cc069d50 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 0x7ff9cc146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 0x7ff9cc146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37530/0 (socket says 172.21.15.79:37530) 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 -- 172.21.15.79:0/1953390909 learned_addr learned my addr 172.21.15.79:0/1953390909 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d2005700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc069d50 0x7ff9cc1f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d2806700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9cc15c1f0 0x7ff9cc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 -- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc069d50 msgr2=0x7ff9cc1f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.405 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc069d50 0x7ff9cc1f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 -- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9cc15c1f0 msgr2=0x7ff9cc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9cc15c1f0 0x7ff9cc0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 -- 172.21.15.79:0/1953390909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c4009a70 con 0x7ff9cc071fe0 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d1804700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 0x7ff9cc146b10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8001eb0 tx=0x7ff9c800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800cea0 con 0x7ff9cc071fe0 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c8005d70 con 0x7ff9cc071fe0 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c00094a0 con 0x7ff9cc071fe0 2023-06-08T03:51:49.406 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80072f0 con 0x7ff9cc071fe0 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.398+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc15de70 con 0x7ff9cc071fe0 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.399+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9cc073e90 con 0x7ff9cc071fe0 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.400+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff9c8007450 con 0x7ff9cc071fe0 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.400+0000 7ff9beffd700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff9b006ace0 0x7ff9b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.400+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6282+0+0 (secure 0 0 0) 0x7ff9c8088900 con 0x7ff9cc071fe0 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.400+0000 7ff9d2005700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff9b006ace0 0x7ff9b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.407 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.401+0000 7ff9d2005700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff9b006ace0 0x7ff9b006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fd0 tx=0x7ff9c001c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:49.408 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.405+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff9c80538d0 con 0x7ff9cc071fe0 2023-06-08T03:51:49.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.628+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff9cc06ed70 con 0x7ff9cc071fe0 2023-06-08T03:51:49.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.636+0000 7ff9beffd700 1 -- 172.21.15.79:0/1953390909 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff9c8056ef0 con 0x7ff9cc071fe0 2023-06-08T03:51:49.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff9b006ace0 msgr2=0x7ff9b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff9b006ace0 0x7ff9b006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009fd0 tx=0x7ff9c001c040 comp rx=0 tx=0).stop 2023-06-08T03:51:49.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 msgr2=0x7ff9cc146b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 0x7ff9cc146b10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8001eb0 tx=0x7ff9c800c7b0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.644 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 shutdown_connections 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9cc15c1f0 0x7ff9cc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7ff9b006ace0 0x7ff9b006d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9cc069d50 0x7ff9cc1f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 --2- 172.21.15.79:0/1953390909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff9cc071fe0 0x7ff9cc146b10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 >> 172.21.15.79:0/1953390909 conn(0x7ff9cc0b1d30 msgr2=0x7ff9cc06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 shutdown_connections 2023-06-08T03:51:49.645 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.640+0000 7ff9d4269700 1 -- 172.21.15.79:0/1953390909 wait complete. 2023-06-08T03:51:49.648 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 6 and data pool 7 2023-06-08T03:51:49.668 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:49.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.783+0000 7f8843a20700 1 Processor -- start 2023-06-08T03:51:49.787 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 -- start start 2023-06-08T03:51:49.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f883c15a640 0x7f883c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f883c15aa10 0x7f883c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c0ce930 0x7f883c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.789 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883c0b0750 con 0x7f883c15aa10 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883c0ceeb0 con 0x7f883c15a640 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.784+0000 7f8843a20700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c0b09a0 con 0x7f883c0ce930 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f883c15aa10 0x7f883c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f883c15aa10 0x7f883c0ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37550/0 (socket says 172.21.15.79:37550) 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 -- 172.21.15.79:0/1766991800 learned_addr learned my addr 172.21.15.79:0/1766991800 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f88417bc700 1 --2- 172.21.15.79:0/1766991800 >> v2:172.21.15.156:3300/0 conn(0x7f883c15a640 0x7f883c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8841fbd700 1 --2- 172.21.15.79:0/1766991800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c0ce930 0x7f883c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.790 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 -- 172.21.15.79:0/1766991800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c0ce930 msgr2=0x7f883c0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 --2- 172.21.15.79:0/1766991800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c0ce930 0x7f883c0af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 -- 172.21.15.79:0/1766991800 >> v2:172.21.15.156:3300/0 conn(0x7f883c15a640 msgr2=0x7f883c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 --2- 172.21.15.79:0/1766991800 >> v2:172.21.15.156:3300/0 conn(0x7f883c15a640 0x7f883c0b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 -- 172.21.15.79:0/1766991800 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0b0c80 con 0x7f883c15aa10 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8840fbb700 1 --2- 172.21.15.79:0/1766991800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15aa10 0x7f883c0ce3f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f883c0b33b0 tx=0x7f882c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325a131225cc7026 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8833fff700 1 -- 172.21.15.79:0/1766991800 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00b870 con 0x7f883c15aa10 2023-06-08T03:51:49.791 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8833fff700 1 -- 172.21.15.79:0/1766991800 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c00b9d0 con 0x7f883c15aa10 2023-06-08T03:51:49.792 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.785+0000 7f8833fff700 1 -- 172.21.15.79:0/1766991800 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00bd60 con 0x7f883c15aa10 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 -- 172.21.15.79:0/1766991800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15aa10 msgr2=0x7f883c0ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 --2- 172.21.15.79:0/1766991800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15aa10 0x7f883c0ce3f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f883c0b33b0 tx=0x7f882c009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 -- 172.21.15.79:0/1766991800 shutdown_connections 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 --2- 172.21.15.79:0/1766991800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c0ce930 0x7f883c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 --2- 172.21.15.79:0/1766991800 >> v2:172.21.15.156:3300/0 conn(0x7f883c15a640 0x7f883c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 --2- 172.21.15.79:0/1766991800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15aa10 0x7f883c0ce3f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 -- 172.21.15.79:0/1766991800 >> 172.21.15.79:0/1766991800 conn(0x7f883c06a670 msgr2=0x7f883c099ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 -- 172.21.15.79:0/1766991800 shutdown_connections 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.786+0000 7f8843a20700 1 -- 172.21.15.79:0/1766991800 wait complete. 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.787+0000 7f8843a20700 1 Processor -- start 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.787+0000 7f8843a20700 1 -- start start 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c0ce930 0x7f883c1f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 0x7f883c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c15aa10 0x7f883c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c0b4090 con 0x7f883c15a640 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c157a90 con 0x7f883c0ce930 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c15b350 con 0x7f883c15aa10 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 0x7f883c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 0x7f883c1572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37558/0 (socket says 172.21.15.79:37558) 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 -- 172.21.15.79:0/93035799 learned_addr learned my addr 172.21.15.79:0/93035799 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f88417bc700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c0ce930 0x7f883c1f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8841fbd700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c15aa10 0x7f883c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 -- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c15aa10 msgr2=0x7f883c152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c15aa10 0x7f883c152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 -- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c0ce930 msgr2=0x7f883c1f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c0ce930 0x7f883c1f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 -- 172.21.15.79:0/93035799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c009550 con 0x7f883c15a640 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8840fbb700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 0x7f883c1572b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f882c000c00 tx=0x7f882c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c004540 con 0x7f883c15a640 2023-06-08T03:51:49.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c0036a0 con 0x7f883c15a640 2023-06-08T03:51:49.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c1577f0 con 0x7f883c15a640 2023-06-08T03:51:49.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.788+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c003a50 con 0x7f883c15a640 2023-06-08T03:51:49.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.789+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c0b3760 con 0x7f883c15a640 2023-06-08T03:51:49.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.789+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c073e90 con 0x7f883c15a640 2023-06-08T03:51:49.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.790+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f882c021070 con 0x7f883c15a640 2023-06-08T03:51:49.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.790+0000 7f8831ffb700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f882006ace0 0x7f882006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:49.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.791+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f882c08acd0 con 0x7f883c15a640 2023-06-08T03:51:49.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.791+0000 7f88417bc700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f882006ace0 0x7f882006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:49.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.791+0000 7f88417bc700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f882006ace0 0x7f882006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8838009930 tx=0x7f8838008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:49.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.795+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f882c055c30 con 0x7f883c15a640 2023-06-08T03:51:49.988 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.984+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f883c0afe90 con 0x7f883c15a640 2023-06-08T03:51:49.989 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.986+0000 7f8831ffb700 1 -- 172.21.15.79:0/93035799 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18427 (secure 0 0 0) 0x7f882c059250 con 0x7f883c15a640 2023-06-08T03:51:49.990 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:49.990 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":37,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:51:49.630602+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:51:47.996030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-06-08T03:51:48.590261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:49.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f882006ace0 msgr2=0x7f882006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f882006ace0 0x7f882006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8838009930 tx=0x7f8838008040 comp rx=0 tx=0).stop 2023-06-08T03:51:49.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 msgr2=0x7f883c1572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:49.992 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 0x7f883c1572b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f882c000c00 tx=0x7f882c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 shutdown_connections 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f882006ace0 0x7f882006d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f883c15aa10 0x7f883c152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c0ce930 0x7f883c1f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 --2- 172.21.15.79:0/93035799 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f883c15a640 0x7f883c1572b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 >> 172.21.15.79:0/93035799 conn(0x7f883c06a670 msgr2=0x7f883c0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 shutdown_connections 2023-06-08T03:51:49.993 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:49.988+0000 7f8843a20700 1 -- 172.21.15.79:0/93035799 wait complete. 2023-06-08T03:51:50.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:50.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.119+0000 7f1a1ac79700 1 Processor -- start 2023-06-08T03:51:50.123 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.119+0000 7f1a1ac79700 1 -- start start 2023-06-08T03:51:50.124 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a1ac79700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1a14071a30 0x7f1a14071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a1ac79700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140cbb10 con 0x7f1a14069d50 2023-06-08T03:51:50.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a1ac79700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a140cbdf0 con 0x7f1a1415a2f0 2023-06-08T03:51:50.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a1ac79700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a140cc0d0 con 0x7f1a14071a30 2023-06-08T03:51:50.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a13fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1a14071a30 0x7f1a14071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a13fff700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1a14071a30 0x7f1a14071ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37564/0 (socket says 172.21.15.79:37564) 2023-06-08T03:51:50.125 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.120+0000 7f1a13fff700 1 -- 172.21.15.79:0/3559920747 learned_addr learned my addr 172.21.15.79:0/3559920747 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a13fff700 1 -- 172.21.15.79:0/3559920747 >> v1:172.21.15.156:6789/0 conn(0x7f1a1415a2f0 legacy=0x7f1a140cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a13fff700 1 -- 172.21.15.79:0/3559920747 >> v1:172.21.15.79:6789/0 conn(0x7f1a14069d50 legacy=0x7f1a1406a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a137fe700 1 -- 172.21.15.79:0/3559920747 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22784340 0 0) 0x7f1a140cbb10 con 0x7f1a14069d50 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a13fff700 1 -- 172.21.15.79:0/3559920747 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a140cc3b0 con 0x7f1a14071a30 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a13fff700 1 --2- 172.21.15.79:0/3559920747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14071a30 0x7f1a14071ea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1a000098f0 tx=0x7f1a00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63153ae4816f1f5 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a137fe700 1 -- 172.21.15.79:0/3559920747 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004230 con 0x7f1a14071a30 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a137fe700 1 -- 172.21.15.79:0/3559920747 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00004390 con 0x7f1a14071a30 2023-06-08T03:51:50.126 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a137fe700 1 -- 172.21.15.79:0/3559920747 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a000036a0 con 0x7f1a14071a30 2023-06-08T03:51:50.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a1ac79700 1 -- 172.21.15.79:0/3559920747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14071a30 msgr2=0x7f1a14071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.127 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/3559920747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14071a30 0x7f1a14071ea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1a000098f0 tx=0x7f1a00009c00 comp rx=0 tx=0).stop 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a1ac79700 1 -- 172.21.15.79:0/3559920747 shutdown_connections 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/3559920747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14071a30 0x7f1a14071ea0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a1ac79700 1 -- 172.21.15.79:0/3559920747 >> 172.21.15.79:0/3559920747 conn(0x7f1a140b1d20 msgr2=0x7f1a1406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.121+0000 7f1a1ac79700 1 -- 172.21.15.79:0/3559920747 shutdown_connections 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.122+0000 7f1a1ac79700 1 -- 172.21.15.79:0/3559920747 wait complete. 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.123+0000 7f1a1ac79700 1 Processor -- start 2023-06-08T03:51:50.128 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.123+0000 7f1a1ac79700 1 -- start start 2023-06-08T03:51:50.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a1ac79700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 0x7f1a14148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.130 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a1ac79700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a14071a30 0x7f1a140c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a1ac79700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a1415a2f0 0x7f1a141452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a1ac79700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a140c7560 con 0x7f1a14069d50 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a1ac79700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14145830 con 0x7f1a1415a2f0 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a1ac79700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14145ad0 con 0x7f1a14071a30 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a18a15700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 0x7f1a14148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a18a15700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 0x7f1a14148700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37570/0 (socket says 172.21.15.79:37570) 2023-06-08T03:51:50.131 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a18a15700 1 -- 172.21.15.79:0/983082424 learned_addr learned my addr 172.21.15.79:0/983082424 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a13fff700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a14071a30 0x7f1a140c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.124+0000 7f1a19216700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a1415a2f0 0x7f1a141452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a18a15700 1 -- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a14071a30 msgr2=0x7f1a140c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a18a15700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a14071a30 0x7f1a140c7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a18a15700 1 -- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a1415a2f0 msgr2=0x7f1a141452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a18a15700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a1415a2f0 0x7f1a141452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a18a15700 1 -- 172.21.15.79:0/983082424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a00009550 con 0x7f1a14069d50 2023-06-08T03:51:50.132 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a13fff700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a14071a30 0x7f1a140c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a19216700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a1415a2f0 0x7f1a141452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a18a15700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 0x7f1a14148700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c4d0 tx=0x7f1a0800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08011790 con 0x7f1a14069d50 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a080118f0 con 0x7f1a14069d50 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a14145db0 con 0x7f1a14069d50 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08011ca0 con 0x7f1a14069d50 2023-06-08T03:51:50.133 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.125+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a14146370 con 0x7f1a14069d50 2023-06-08T03:51:50.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.126+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a14073e90 con 0x7f1a14069d50 2023-06-08T03:51:50.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.126+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a0801c460 con 0x7f1a14069d50 2023-06-08T03:51:50.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.127+0000 7f1a117fa700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19f806ace0 0x7f19f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.127+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1a08089470 con 0x7f1a14069d50 2023-06-08T03:51:50.134 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.127+0000 7f1a13fff700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19f806ace0 0x7f19f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.127+0000 7f1a13fff700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19f806ace0 0x7f19f806d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009130 tx=0x7f1a00008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:50.135 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.131+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a080543e0 con 0x7f1a14069d50 2023-06-08T03:51:50.325 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.322+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a140726e0 con 0x7f1a14069d50 2023-06-08T03:51:50.327 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.323+0000 7f1a117fa700 1 -- 172.21.15.79:0/983082424 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18427 (secure 0 0 0) 0x7f1a08057a00 con 0x7f1a14069d50 2023-06-08T03:51:50.327 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:50.327 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":37,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:51:49.630602+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:51:47.996030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-06-08T03:51:48.590261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:50.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19f806ace0 msgr2=0x7f19f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19f806ace0 0x7f19f806d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009130 tx=0x7f1a00008030 comp rx=0 tx=0).stop 2023-06-08T03:51:50.329 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 msgr2=0x7f1a14148700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 0x7f1a14148700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c4d0 tx=0x7f1a0800c890 comp rx=0 tx=0).stop 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 shutdown_connections 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a1415a2f0 0x7f1a141452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a14069d50 0x7f1a14148700 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f19f806ace0 0x7f19f806d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 --2- 172.21.15.79:0/983082424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a14071a30 0x7f1a140c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.330 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 >> 172.21.15.79:0/983082424 conn(0x7f1a140b1d20 msgr2=0x7f1a14072900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:50.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 shutdown_connections 2023-06-08T03:51:50.331 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.325+0000 7f1a1ac79700 1 -- 172.21.15.79:0/983082424 wait complete. 2023-06-08T03:51:50.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-08T03:51:50.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.459+0000 7fb7e6cbd700 1 Processor -- start 2023-06-08T03:51:50.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.460+0000 7fb7e6cbd700 1 -- start start 2023-06-08T03:51:50.466 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e6cbd700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb7e0071070 0x7fb7e015d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e6cbd700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e015d8d0 con 0x7fb7e0158ec0 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e6cbd700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e00ca320 con 0x7fb7e00cc650 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e6cbd700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e006df90 con 0x7fb7e0071070 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e4a59700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb7e0158ec0 0x7fb7e00cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32822/0 (socket says 172.21.15.79:32822) 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e4a59700 1 -- 172.21.15.79:0/3042206575 learned_addr learned my addr 172.21.15.79:0/3042206575 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3042206575 >> v2:172.21.15.156:3300/0 conn(0x7fb7e0071070 0x7fb7e015d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.467 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793496099 0 0) 0x7fb7d8001490 con 0x7fb7e0158ec0 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7cc0035b0 con 0x7fb7e0158ec0 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 432776072 0 0) 0x7fb7cc0035b0 con 0x7fb7e0158ec0 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 >> v2:172.21.15.156:3300/0 conn(0x7fb7e0071070 msgr2=0x7fb7e015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 --2- 172.21.15.79:0/3042206575 >> v2:172.21.15.156:3300/0 conn(0x7fb7e0071070 0x7fb7e015d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 >> v1:172.21.15.156:6789/0 conn(0x7fb7e00cc650 legacy=0x7fb7e00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0159290 con 0x7fb7e0158ec0 2023-06-08T03:51:50.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fb7d8001b80 con 0x7fb7e0158ec0 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 >> v1:172.21.15.79:6789/0 conn(0x7fb7e0158ec0 legacy=0x7fb7e00cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7cc003a00 0x7fb7cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc004850 0x7fb7cc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc005200 0x7fb7cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e006df90 con 0x7fb7cc005200 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc00c080 con 0x7fb7cc004850 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.461+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc00c240 con 0x7fb7cc003a00 2023-06-08T03:51:50.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc005200 0x7fb7cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7dffff700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc004850 0x7fb7cc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e4a59700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7cc003a00 0x7fb7cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 -- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7cc003a00 msgr2=0x7fb7cc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7cc003a00 0x7fb7cc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 -- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc004850 msgr2=0x7fb7cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc004850 0x7fb7cc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 -- 172.21.15.79:0/3042206575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7cc0035b0 con 0x7fb7cc005200 2023-06-08T03:51:50.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 -- 172.21.15.79:0/3042206575 reap_dead start 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc005200 0x7fb7cc00ba10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009b30 tx=0x7fb7cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239306e285068cc3 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0004020 con 0x7fb7cc005200 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d0004180 con 0x7fb7cc005200 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.462+0000 7fb7df7fe700 1 -- 172.21.15.79:0/3042206575 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0004530 con 0x7fb7cc005200 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3042206575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc005200 msgr2=0x7fb7cc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc005200 0x7fb7cc00ba10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009b30 tx=0x7fb7cc005880 comp rx=0 tx=0).stop 2023-06-08T03:51:50.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3042206575 shutdown_connections 2023-06-08T03:51:50.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3042206575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7cc005200 0x7fb7cc00ba10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3042206575 >> 172.21.15.79:0/3042206575 conn(0x7fb7e00b1d30 msgr2=0x7fb7e006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:50.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3042206575 shutdown_connections 2023-06-08T03:51:50.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.463+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3042206575 wait complete. 2023-06-08T03:51:50.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.464+0000 7fb7e6cbd700 1 Processor -- start 2023-06-08T03:51:50.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.464+0000 7fb7e6cbd700 1 -- start start 2023-06-08T03:51:50.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e6cbd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc005200 0x7fb7e014cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e6cbd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7e00c8850 0x7fb7e00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e6cbd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 0x7fb7e00c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e6cbd700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c81f0 con 0x7fb7e00c6800 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e6cbd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c9200 con 0x7fb7cc005200 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e6cbd700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e014d140 con 0x7fb7e00c8850 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e525a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 0x7fb7e00c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e525a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 0x7fb7e00c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37594/0 (socket says 172.21.15.79:37594) 2023-06-08T03:51:50.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.465+0000 7fb7e525a700 1 -- 172.21.15.79:0/3013528945 learned_addr learned my addr 172.21.15.79:0/3013528945 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7dffff700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7e00c8850 0x7fb7e00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e525a700 1 -- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7e00c8850 msgr2=0x7fb7e00c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e4a59700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc005200 0x7fb7e014cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7e00c8850 0x7fb7e00c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e525a700 1 -- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc005200 msgr2=0x7fb7e014cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc005200 0x7fb7e014cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:50.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e525a700 1 -- 172.21.15.79:0/3013528945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00c7570 con 0x7fb7e00c6800 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7dffff700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7e00c8850 0x7fb7e00c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e4a59700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc005200 0x7fb7e014cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e525a700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 0x7fb7e00c6c70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0071000 tx=0x7fb7d00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0020070 con 0x7fb7e00c6800 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d000eda0 con 0x7fb7e00c6800 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0004020 con 0x7fb7e00c6800 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e00c7850 con 0x7fb7e00c6800 2023-06-08T03:51:50.477 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.466+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e00c7e10 con 0x7fb7e00c6800 2023-06-08T03:51:50.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.467+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e00c7fd0 con 0x7fb7e00c6800 2023-06-08T03:51:50.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.467+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7d0024070 con 0x7fb7e00c6800 2023-06-08T03:51:50.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.468+0000 7fb7dd7fa700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb7c006ace0 0x7fb7c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:50.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.468+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb7d008a1a0 con 0x7fb7e00c6800 2023-06-08T03:51:50.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.468+0000 7fb7e4a59700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb7c006ace0 0x7fb7c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:50.478 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.468+0000 7fb7e4a59700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb7c006ace0 0x7fb7c006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80016f0 tx=0x7fb7d8006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:50.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.472+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb7d00552c0 con 0x7fb7e00c6800 2023-06-08T03:51:50.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:50.704+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7e00c8180 con 0x7fb7e00c6800 2023-06-08T03:51:51.647 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.643+0000 7fb7dd7fa700 1 -- 172.21.15.79:0/3013528945 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb7d00588e0 con 0x7fb7e00c6800 2023-06-08T03:51:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb7c006ace0 msgr2=0x7fb7c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb7c006ace0 0x7fb7c006d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb7d80016f0 tx=0x7fb7d8006040 comp rx=0 tx=0).stop 2023-06-08T03:51:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 msgr2=0x7fb7e00c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:51.650 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 0x7fb7e00c6c70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0071000 tx=0x7fb7d00047c0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 shutdown_connections 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb7e00c6800 0x7fb7e00c6c70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb7cc005200 0x7fb7e014cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fb7c006ace0 0x7fb7c006d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 --2- 172.21.15.79:0/3013528945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb7e00c8850 0x7fb7e00c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 >> 172.21.15.79:0/3013528945 conn(0x7fb7e00b1d30 msgr2=0x7fb7e00ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 shutdown_connections 2023-06-08T03:51:51.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.646+0000 7fb7e6cbd700 1 -- 172.21.15.79:0/3013528945 wait complete. 2023-06-08T03:51:51.664 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-08T03:51:51.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-08T03:51:51.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.789+0000 7fa992642700 1 Processor -- start 2023-06-08T03:51:51.793 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.789+0000 7fa992642700 1 -- start start 2023-06-08T03:51:51.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa992642700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa98c0d0060 0x7fa98c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:51.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa992642700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa98c0cf490 0x7fa98c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa992642700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c0cae50 con 0x7fa98c0cd080 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa992642700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0cbea0 con 0x7fa98c0cf490 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa992642700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0cda30 con 0x7fa98c0d0060 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa98c0cf490 0x7fa98c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa98c0cf490 0x7fa98c0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37602/0 (socket says 172.21.15.79:37602) 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 -- 172.21.15.79:0/1442616096 learned_addr learned my addr 172.21.15.79:0/1442616096 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:51.795 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 -- 172.21.15.79:0/1442616096 >> v2:172.21.15.156:3300/0 conn(0x7fa98c0d0060 msgr2=0x7fa98c0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 --2- 172.21.15.79:0/1442616096 >> v2:172.21.15.156:3300/0 conn(0x7fa98c0d0060 0x7fa98c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 -- 172.21.15.79:0/1442616096 >> v1:172.21.15.156:6789/0 conn(0x7fa98c0cd080 legacy=0x7fa98c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 -- 172.21.15.79:0/1442616096 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c15d930 con 0x7fa98c0cf490 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa990bdf700 1 --2- 172.21.15.79:0/1442616096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c0cd4f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa98c15d890 tx=0x7fa98400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffd27724a60334 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa98affd700 1 -- 172.21.15.79:0/1442616096 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98400e070 con 0x7fa98c0cf490 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa98affd700 1 -- 172.21.15.79:0/1442616096 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984003a20 con 0x7fa98c0cf490 2023-06-08T03:51:51.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.790+0000 7fa98affd700 1 -- 172.21.15.79:0/1442616096 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984003dd0 con 0x7fa98c0cf490 2023-06-08T03:51:51.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 -- 172.21.15.79:0/1442616096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 msgr2=0x7fa98c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:51.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 --2- 172.21.15.79:0/1442616096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c0cd4f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa98c15d890 tx=0x7fa98400b330 comp rx=0 tx=0).stop 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 -- 172.21.15.79:0/1442616096 shutdown_connections 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 --2- 172.21.15.79:0/1442616096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c0cd4f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 --2- 172.21.15.79:0/1442616096 >> v2:172.21.15.156:3300/0 conn(0x7fa98c0d0060 0x7fa98c0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 -- 172.21.15.79:0/1442616096 >> 172.21.15.79:0/1442616096 conn(0x7fa98c06e1e0 msgr2=0x7fa98c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 -- 172.21.15.79:0/1442616096 shutdown_connections 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.791+0000 7fa992642700 1 -- 172.21.15.79:0/1442616096 wait complete. 2023-06-08T03:51:51.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.792+0000 7fa992642700 1 Processor -- start 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.792+0000 7fa992642700 1 -- start start 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa992642700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa98c0cd080 0x7fa98c0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa992642700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa992642700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa98c0d0060 0x7fa98c144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa992642700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c144680 con 0x7fa98c0cf490 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa992642700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c96b0 con 0x7fa98c0cd080 2023-06-08T03:51:51.800 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa992642700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c0c9990 con 0x7fa98c0d0060 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa98b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa98b7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37606/0 (socket says 172.21.15.79:37606) 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa98b7fe700 1 -- 172.21.15.79:0/540755918 learned_addr learned my addr 172.21.15.79:0/540755918 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa98bfff700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa98c0cd080 0x7fa98c0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.793+0000 7fa990bdf700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa98c0d0060 0x7fa98c144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa98b7fe700 1 -- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa98c0d0060 msgr2=0x7fa98c144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa98b7fe700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa98c0d0060 0x7fa98c144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa98b7fe700 1 -- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa98c0cd080 msgr2=0x7fa98c0c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa98b7fe700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa98c0cd080 0x7fa98c0c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa98b7fe700 1 -- 172.21.15.79:0/540755918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009b30 con 0x7fa98c0cf490 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa98b7fe700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c159590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b450 tx=0x7fa97c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011830 con 0x7fa98c0cf490 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c011990 con 0x7fa98c0cf490 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c0c9c70 con 0x7fa98c0cf490 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c011d20 con 0x7fa98c0cf490 2023-06-08T03:51:51.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.794+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c0ca230 con 0x7fa98c0cf490 2023-06-08T03:51:51.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.795+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c073e90 con 0x7fa98c0cf490 2023-06-08T03:51:51.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.795+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa97c01a400 con 0x7fa98c0cf490 2023-06-08T03:51:51.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.796+0000 7fa988ff9700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa97806ace0 0x7fa97806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:51.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.796+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa97c089410 con 0x7fa98c0cf490 2023-06-08T03:51:51.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.796+0000 7fa98bfff700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa97806ace0 0x7fa97806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:51.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.797+0000 7fa98bfff700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa97806ace0 0x7fa97806d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa980009a10 tx=0x7fa980009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:51.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:51.800+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa97c054410 con 0x7fa98c0cf490 2023-06-08T03:51:52.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.032+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa98c070120 con 0x7fa98c0cf490 2023-06-08T03:51:52.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.650+0000 7fa988ff9700 1 -- 172.21.15.79:0/540755918 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa97c00be60 con 0x7fa98c0cf490 2023-06-08T03:51:52.656 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa97806ace0 msgr2=0x7fa97806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:52.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa97806ace0 0x7fa97806d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa980009a10 tx=0x7fa980009190 comp rx=0 tx=0).stop 2023-06-08T03:51:52.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 msgr2=0x7fa98c159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:52.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c159590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b450 tx=0x7fa97c00b760 comp rx=0 tx=0).stop 2023-06-08T03:51:52.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 shutdown_connections 2023-06-08T03:51:52.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fa97806ace0 0x7fa97806d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.657 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa98c0d0060 0x7fa98c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa98c0cd080 0x7fa98c0c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 --2- 172.21.15.79:0/540755918 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa98c0cf490 0x7fa98c159590 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 >> 172.21.15.79:0/540755918 conn(0x7fa98c06e1e0 msgr2=0x7fa98c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:52.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.652+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 shutdown_connections 2023-06-08T03:51:52.658 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.653+0000 7fa992642700 1 -- 172.21.15.79:0/540755918 wait complete. 2023-06-08T03:51:52.670 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-08T03:51:52.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:51:52.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.790+0000 7f03b1c61700 1 Processor -- start 2023-06-08T03:51:52.794 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.790+0000 7f03b1c61700 1 -- start start 2023-06-08T03:51:52.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03b1c61700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f03ac0b1cd0 0x7f03ac0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:52.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03b1c61700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f03ac0cd060 0x7f03ac0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:52.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03b1c61700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac0cad20 con 0x7f03ac0afb30 2023-06-08T03:51:52.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03b1c61700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0cb8a0 con 0x7f03ac0cd060 2023-06-08T03:51:52.796 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03b1c61700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0caf70 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aaffd700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f03ac0cd060 0x7f03ac0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03abfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f03ac0afb30 0x7f03ac15dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32860/0 (socket says 172.21.15.79:32860) 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03abfff700 1 -- 172.21.15.79:0/2819395272 learned_addr learned my addr 172.21.15.79:0/2819395272 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aaffd700 1 -- 172.21.15.79:0/2819395272 >> v2:172.21.15.156:3300/0 conn(0x7f03ac0b1cd0 msgr2=0x7f03ac0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03ab7fe700 1 --2- 172.21.15.79:0/2819395272 >> v2:172.21.15.156:3300/0 conn(0x7f03ac0b1cd0 0x7f03ac0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aaffd700 1 --2- 172.21.15.79:0/2819395272 >> v2:172.21.15.156:3300/0 conn(0x7f03ac0b1cd0 0x7f03ac0ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aaffd700 1 -- 172.21.15.79:0/2819395272 >> v1:172.21.15.79:6789/0 conn(0x7f03ac0afb30 legacy=0x7f03ac15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:52.797 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aa7fc700 1 -- 172.21.15.79:0/2819395272 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475103369 0 0) 0x7f03ac0cad20 con 0x7f03ac0afb30 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aaffd700 1 -- 172.21.15.79:0/2819395272 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0cb250 con 0x7f03ac0cd060 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aaffd700 1 --2- 172.21.15.79:0/2819395272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0cd060 0x7f03ac0ce7e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f03ac071550 tx=0x7f039c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c041c7383bf893e2 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aa7fc700 1 -- 172.21.15.79:0/2819395272 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c011830 con 0x7f03ac0cd060 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aa7fc700 1 -- 172.21.15.79:0/2819395272 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c011990 con 0x7f03ac0cd060 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.791+0000 7f03aa7fc700 1 -- 172.21.15.79:0/2819395272 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c011d20 con 0x7f03ac0cd060 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 -- 172.21.15.79:0/2819395272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0cd060 msgr2=0x7f03ac0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:52.798 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 --2- 172.21.15.79:0/2819395272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0cd060 0x7f03ac0ce7e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f03ac071550 tx=0x7f039c009b20 comp rx=0 tx=0).stop 2023-06-08T03:51:52.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 -- 172.21.15.79:0/2819395272 shutdown_connections 2023-06-08T03:51:52.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 --2- 172.21.15.79:0/2819395272 >> v2:172.21.15.156:3300/0 conn(0x7f03ac0b1cd0 0x7f03ac0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 --2- 172.21.15.79:0/2819395272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0cd060 0x7f03ac0ce7e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 -- 172.21.15.79:0/2819395272 >> 172.21.15.79:0/2819395272 conn(0x7f03ac0999b0 msgr2=0x7f03ac0d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:52.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 -- 172.21.15.79:0/2819395272 shutdown_connections 2023-06-08T03:51:52.799 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.792+0000 7f03b1c61700 1 -- 172.21.15.79:0/2819395272 wait complete. 2023-06-08T03:51:52.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.793+0000 7f03b1c61700 1 Processor -- start 2023-06-08T03:51:52.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.793+0000 7f03b1c61700 1 -- start start 2023-06-08T03:51:52.801 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03b1c61700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03ac0afb30 0x7f03ac0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03b1c61700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 0x7f03ac06c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03b1c61700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03ac0cd060 0x7f03ac157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03b1c61700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0c9500 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03b1c61700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0c7c30 con 0x7f03ac0cd060 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03b1c61700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0c7f10 con 0x7f03ac0afb30 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03aaffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 0x7f03ac06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:52.802 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03aaffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 0x7f03ac06c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37620/0 (socket says 172.21.15.79:37620) 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03aaffd700 1 -- 172.21.15.79:0/3817431556 learned_addr learned my addr 172.21.15.79:0/3817431556 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03abfff700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03ac0cd060 0x7f03ac157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.794+0000 7f03ab7fe700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03ac0afb30 0x7f03ac0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03aaffd700 1 -- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03ac0afb30 msgr2=0x7f03ac0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03aaffd700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03ac0afb30 0x7f03ac0c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03aaffd700 1 -- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03ac0cd060 msgr2=0x7f03ac157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03aaffd700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03ac0cd060 0x7f03ac157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:52.803 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03aaffd700 1 -- 172.21.15.79:0/3817431556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0094a0 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03aaffd700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 0x7f03ac06c850 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f039c014040 tx=0x7f039c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0038a0 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c01e070 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac0c81f0 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c003c40 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.795+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac0c87b0 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.804 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.796+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac073e90 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.796+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f039c02d450 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.797+0000 7f0393fff700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f039806ace0 0x7f039806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:52.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.797+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f039c08a6c0 con 0x7f03ac0b1cd0 2023-06-08T03:51:52.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.797+0000 7f03ab7fe700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f039806ace0 0x7f039806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:52.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.797+0000 7f03ab7fe700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f039806ace0 0x7f039806d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03a000a990 tx=0x7f03a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:52.805 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:52.801+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f039c0556c0 con 0x7f03ac0b1cd0 2023-06-08T03:51:53.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.027+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03ac08aa20 con 0x7f03ac0b1cd0 2023-06-08T03:51:53.031 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.028+0000 7f0393fff700 1 -- 172.21.15.79:0/3817431556 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4677 (secure 0 0 0) 0x7f039c058ce0 con 0x7f03ac0b1cd0 2023-06-08T03:51:53.032 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:53.032 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/3221038422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3221038422},{"type":"v1","addr":"172.21.15.156:6836","nonce":3221038422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2525455221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2525455221},{"type":"v1","addr":"172.21.15.156:6837","nonce":2525455221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6834/3327951478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3327951478},{"type":"v1","addr":"172.21.15.79:6834","nonce":3327951478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:51:49.630919+0000","modified":"2023-06-08T03:51:52.642991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6835/2809764197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6833","nonce":2809764197},{"type":"v1","addr":"172.21.15.79:6835","nonce":2809764197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.029+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f039806ace0 msgr2=0x7f039806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.029+0000 7f03b1c61700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f039806ace0 0x7f039806d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f03a000a990 tx=0x7f03a0008040 comp rx=0 tx=0).stop 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.029+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 msgr2=0x7f03ac06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.029+0000 7f03b1c61700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 0x7f03ac06c850 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f039c014040 tx=0x7f039c004750 comp rx=0 tx=0).stop 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 shutdown_connections 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03ac0cd060 0x7f03ac157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.034 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f039806ace0 0x7f039806d190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f03ac0afb30 0x7f03ac0c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 --2- 172.21.15.79:0/3817431556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03ac0b1cd0 0x7f03ac06c850 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 >> 172.21.15.79:0/3817431556 conn(0x7f03ac0999b0 msgr2=0x7f03ac0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:53.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 shutdown_connections 2023-06-08T03:51:53.035 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.030+0000 7f03b1c61700 1 -- 172.21.15.79:0/3817431556 wait complete. 2023-06-08T03:51:53.036 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 17 2023-06-08T03:51:53.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-08T03:51:53.165 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.161+0000 7f7a457eb700 1 Processor -- start 2023-06-08T03:51:53.166 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.161+0000 7f7a457eb700 1 -- start start 2023-06-08T03:51:53.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.162+0000 7f7a457eb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400c9630 0x7f7a40158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.162+0000 7f7a457eb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a40158f80 0x7f7a400ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.162+0000 7f7a457eb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a40159a10 con 0x7f7a400cd9b0 2023-06-08T03:51:53.167 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.162+0000 7f7a457eb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a400cc810 con 0x7f7a40158f80 2023-06-08T03:51:53.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.162+0000 7f7a457eb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40159450 con 0x7f7a400c9630 2023-06-08T03:51:53.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3effd700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400c9630 0x7f7a40158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3e7fc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a40158f80 0x7f7a400ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3f7fe700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f7a400cd9b0 0x7f7a40159c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:35034/0 (socket says 172.21.15.79:35034) 2023-06-08T03:51:53.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3f7fe700 1 -- 172.21.15.79:0/262733538 learned_addr learned my addr 172.21.15.79:0/262733538 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:53.168 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3dffb700 1 -- 172.21.15.79:0/262733538 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859446329 0 0) 0x7f7a40159a10 con 0x7f7a400cd9b0 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3dffb700 1 -- 172.21.15.79:0/262733538 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a280035b0 con 0x7f7a400cd9b0 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3effd700 1 -- 172.21.15.79:0/262733538 >> v2:172.21.15.156:3300/0 conn(0x7f7a40158f80 msgr2=0x7f7a400ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3effd700 1 --2- 172.21.15.79:0/262733538 >> v2:172.21.15.156:3300/0 conn(0x7f7a40158f80 0x7f7a400ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3effd700 1 -- 172.21.15.79:0/262733538 >> v1:172.21.15.156:6789/0 conn(0x7f7a400cd9b0 legacy=0x7f7a40159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3effd700 1 -- 172.21.15.79:0/262733538 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40159730 con 0x7f7a400c9630 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3e7fc700 1 --2- 172.21.15.79:0/262733538 >> v2:172.21.15.156:3300/0 conn(0x7f7a40158f80 0x7f7a400ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3effd700 1 --2- 172.21.15.79:0/262733538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400c9630 0x7f7a40158a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009840 tx=0x7f7a34009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d60c2d2e1aa995e server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.169 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3dffb700 1 -- 172.21.15.79:0/262733538 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34011830 con 0x7f7a400c9630 2023-06-08T03:51:53.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3dffb700 1 -- 172.21.15.79:0/262733538 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a34011990 con 0x7f7a400c9630 2023-06-08T03:51:53.170 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.163+0000 7f7a3dffb700 1 -- 172.21.15.79:0/262733538 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34011d20 con 0x7f7a400c9630 2023-06-08T03:51:53.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 -- 172.21.15.79:0/262733538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400c9630 msgr2=0x7f7a40158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 --2- 172.21.15.79:0/262733538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400c9630 0x7f7a40158a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009840 tx=0x7f7a34009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:53.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 -- 172.21.15.79:0/262733538 shutdown_connections 2023-06-08T03:51:53.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 --2- 172.21.15.79:0/262733538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400c9630 0x7f7a40158a40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 --2- 172.21.15.79:0/262733538 >> v2:172.21.15.156:3300/0 conn(0x7f7a40158f80 0x7f7a400ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.172 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 -- 172.21.15.79:0/262733538 >> 172.21.15.79:0/262733538 conn(0x7f7a40069d50 msgr2=0x7f7a4006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 -- 172.21.15.79:0/262733538 shutdown_connections 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.164+0000 7f7a457eb700 1 -- 172.21.15.79:0/262733538 wait complete. 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.165+0000 7f7a457eb700 1 Processor -- start 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.165+0000 7f7a457eb700 1 -- start start 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a457eb700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 0x7f7a400c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a457eb700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400cd9b0 0x7f7a400b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a457eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40158f80 0x7f7a400c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.173 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a457eb700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a401f2c00 con 0x7f7a400c9630 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a457eb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400c7a70 con 0x7f7a40158f80 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a457eb700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400c7d50 con 0x7f7a400cd9b0 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 0x7f7a400c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 0x7f7a400c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37630/0 (socket says 172.21.15.79:37630) 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 -- 172.21.15.79:0/1785596388 learned_addr learned my addr 172.21.15.79:0/1785596388 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3e7fc700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400cd9b0 0x7f7a400b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3f7fe700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40158f80 0x7f7a400c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.174 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 -- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400cd9b0 msgr2=0x7f7a400b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400cd9b0 0x7f7a400b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 -- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40158f80 msgr2=0x7f7a400c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40158f80 0x7f7a400c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3effd700 1 -- 172.21.15.79:0/1785596388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300094a0 con 0x7f7a400c9630 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.166+0000 7f7a3f7fe700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40158f80 0x7f7a400c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.167+0000 7f7a3effd700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 0x7f7a400c73e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7a340090b0 tx=0x7f7a34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.175 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.167+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34004550 con 0x7f7a400c9630 2023-06-08T03:51:53.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.167+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3401f070 con 0x7f7a400c9630 2023-06-08T03:51:53.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.167+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a340038e0 con 0x7f7a400c9630 2023-06-08T03:51:53.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.167+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a340094a0 con 0x7f7a400c9630 2023-06-08T03:51:53.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.167+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a401f2e50 con 0x7f7a400c9630 2023-06-08T03:51:53.176 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.168+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a40073e90 con 0x7f7a400c9630 2023-06-08T03:51:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.168+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a34003a40 con 0x7f7a400c9630 2023-06-08T03:51:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.169+0000 7f7a277fe700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a2c06ace0 0x7f7a2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.169+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7a34021070 con 0x7f7a400c9630 2023-06-08T03:51:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.169+0000 7f7a3e7fc700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a2c06ace0 0x7f7a2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.177 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.170+0000 7f7a3e7fc700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a2c06ace0 0x7f7a2c06d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a400b3a80 tx=0x7f7a30009210 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.178 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.173+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7a34055db0 con 0x7f7a400c9630 2023-06-08T03:51:53.217 INFO:tasks.ceph.mds.c.smithi079.stderr:2023-06-08T03:51:53.214+0000 7f7ff5ad7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-08T03:51:53.362 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.359+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a40159db0 con 0x7f7a400c9630 2023-06-08T03:51:53.364 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.360+0000 7f7a277fe700 1 -- 172.21.15.79:0/1785596388 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18414 (secure 0 0 0) 0x7f7a340593d0 con 0x7f7a400c9630 2023-06-08T03:51:53.364 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:53.364 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":38,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","created":"2023-06-08T03:50:06.150869+0000","modified":"2023-06-08T03:51:50.632669+0000","last_up_change":"2023-06-08T03:50:16.138854+0000","last_in_change":"2023-06-08T03:50:08.640471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-08T03:50:17.429601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-08T03:50:17.909779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-06-08T03:51:47.996030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-06-08T03:51:48.590261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ca6ef60-f85f-4f89-8729-e7e64956e1a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6801","nonce":233623149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6803","nonce":233623149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6807","nonce":233623149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":233623149},{"type":"v1","addr":"172.21.15.79:6805","nonce":233623149}]},"public_addr":"172.21.15.79:6801/233623149","cluster_addr":"172.21.15.79:6803/233623149","heartbeat_back_addr":"172.21.15.79:6807/233623149","heartbeat_front_addr":"172.21.15.79:6805/233623149","state":["exists","up"]},{"osd":1,"uuid":"e5277170-8966-49c0-8c10-32be4c9d1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6817","nonce":4207501459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6819","nonce":4207501459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6823","nonce":4207501459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":4207501459},{"type":"v1","addr":"172.21.15.79:6821","nonce":4207501459}]},"public_addr":"172.21.15.79:6817/4207501459","cluster_addr":"172.21.15.79:6819/4207501459","heartbeat_back_addr":"172.21.15.79:6823/4207501459","heartbeat_front_addr":"172.21.15.79:6821/4207501459","state":["exists","up"]},{"osd":2,"uuid":"b3f37967-178c-4d15-9441-a4b57cd8b602","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6809","nonce":4041064949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6811","nonce":4041064949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6815","nonce":4041064949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":4041064949},{"type":"v1","addr":"172.21.15.79:6813","nonce":4041064949}]},"public_addr":"172.21.15.79:6809/4041064949","cluster_addr":"172.21.15.79:6811/4041064949","heartbeat_back_addr":"172.21.15.79:6815/4041064949","heartbeat_front_addr":"172.21.15.79:6813/4041064949","state":["exists","up"]},{"osd":3,"uuid":"a6ab1aab-9727-482c-8b65-faf5925615a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6825","nonce":481813105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6827","nonce":481813105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6831","nonce":481813105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":481813105},{"type":"v1","addr":"172.21.15.79:6829","nonce":481813105}]},"public_addr":"172.21.15.79:6825/481813105","cluster_addr":"172.21.15.79:6827/481813105","heartbeat_back_addr":"172.21.15.79:6831/481813105","heartbeat_front_addr":"172.21.15.79:6829/481813105","state":["exists","up"]},{"osd":4,"uuid":"b6d3f147-10af-4fb9-94ee-49b2b58dc603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6801","nonce":1078658928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6803","nonce":1078658928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6807","nonce":1078658928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":1078658928},{"type":"v1","addr":"172.21.15.156:6805","nonce":1078658928}]},"public_addr":"172.21.15.156:6801/1078658928","cluster_addr":"172.21.15.156:6803/1078658928","heartbeat_back_addr":"172.21.15.156:6807/1078658928","heartbeat_front_addr":"172.21.15.156:6805/1078658928","state":["exists","up"]},{"osd":5,"uuid":"8b340c4a-a17d-428b-8182-fa1a28f3d274","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6809","nonce":353543302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6811","nonce":353543302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6815","nonce":353543302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":353543302},{"type":"v1","addr":"172.21.15.156:6813","nonce":353543302}]},"public_addr":"172.21.15.156:6809/353543302","cluster_addr":"172.21.15.156:6811/353543302","heartbeat_back_addr":"172.21.15.156:6815/353543302","heartbeat_front_addr":"172.21.15.156:6813/353543302","state":["exists","up"]},{"osd":6,"uuid":"318c708a-4f08-4506-9c31-7f0b3f59b582","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6825","nonce":3611210445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6827","nonce":3611210445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6831","nonce":3611210445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":3611210445},{"type":"v1","addr":"172.21.15.156:6829","nonce":3611210445}]},"public_addr":"172.21.15.156:6825/3611210445","cluster_addr":"172.21.15.156:6827/3611210445","heartbeat_back_addr":"172.21.15.156:6831/3611210445","heartbeat_front_addr":"172.21.15.156:6829/3611210445","state":["exists","up"]},{"osd":7,"uuid":"735d6453-9a2d-45cd-9877-3ad4133082d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6817","nonce":3505887928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6819","nonce":3505887928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6823","nonce":3505887928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3505887928},{"type":"v1","addr":"172.21.15.156:6821","nonce":3505887928}]},"public_addr":"172.21.15.156:6817/3505887928","cluster_addr":"172.21.15.156:6819/3505887928","heartbeat_back_addr":"172.21.15.156:6823/3505887928","heartbeat_front_addr":"172.21.15.156:6821/3505887928","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.977426+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:16.014676+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:15.976423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.848330+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.888155+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.869878+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.874190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-08T03:50:13.849772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-08T03:51:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a2c06ace0 msgr2=0x7f7a2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a2c06ace0 0x7f7a2c06d190 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a400b3a80 tx=0x7f7a30009210 comp rx=0 tx=0).stop 2023-06-08T03:51:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 msgr2=0x7f7a400c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.366 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 0x7f7a400c73e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7a340090b0 tx=0x7f7a34004750 comp rx=0 tx=0).stop 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 shutdown_connections 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a40158f80 0x7f7a400c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a400c9630 0x7f7a400c73e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f7a2c06ace0 0x7f7a2c06d190 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 --2- 172.21.15.79:0/1785596388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a400cd9b0 0x7f7a400b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 >> 172.21.15.79:0/1785596388 conn(0x7f7a40069d50 msgr2=0x7f7a4015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 shutdown_connections 2023-06-08T03:51:53.367 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.362+0000 7f7a457eb700 1 -- 172.21.15.79:0/1785596388 wait complete. 2023-06-08T03:51:53.378 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-08T03:51:53.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-08T03:51:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.493+0000 7fbd467fc700 1 Processor -- start 2023-06-08T03:51:53.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.493+0000 7fbd467fc700 1 -- start start 2023-06-08T03:51:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd467fc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 0x7fbd4006f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd467fc700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4006edd0 con 0x7fbd4015c680 2023-06-08T03:51:53.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd467fc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4006eaa0 con 0x7fbd400b0fe0 2023-06-08T03:51:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd467fc700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4006f7a0 con 0x7fbd4006bb30 2023-06-08T03:51:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd3ffff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbd4015c680 0x7fbd400ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32882/0 (socket says 172.21.15.79:32882) 2023-06-08T03:51:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd3ffff700 1 -- 172.21.15.79:0/3535448958 learned_addr learned my addr 172.21.15.79:0/3535448958 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101789276 0 0) 0x7fbd4006edd0 con 0x7fbd4015c680 2023-06-08T03:51:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd280035b0 con 0x7fbd4015c680 2023-06-08T03:51:53.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.494+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 0x7fbd4006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607349314 0 0) 0x7fbd280035b0 con 0x7fbd4015c680 2023-06-08T03:51:53.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 msgr2=0x7fbd4006f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 0x7fbd4006f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 >> v1:172.21.15.156:6789/0 conn(0x7fbd400b0fe0 legacy=0x7fbd4015ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd40069e80 con 0x7fbd4015c680 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fbd34001670 con 0x7fbd4015c680 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 >> v1:172.21.15.79:6789/0 conn(0x7fbd4015c680 legacy=0x7fbd400ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28003aa0 0x7fbd28003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd28004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 0x7fbd2800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4006f7a0 con 0x7fbd28004850 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd28005530 con 0x7fbd28003aa0 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280056f0 con 0x7fbd4006bb30 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd28004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3ffff700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28003aa0 0x7fbd28003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd44d99700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 0x7fbd2800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 -- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 msgr2=0x7fbd2800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd4006bb30 0x7fbd2800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 -- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28003aa0 msgr2=0x7fbd28003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd28003aa0 0x7fbd28003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 -- 172.21.15.79:0/3535448958 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd280035b0 con 0x7fbd28004850 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd28004cc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009b60 tx=0x7fbd28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d08af19361ec46f server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.495+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000e6c0 con 0x7fbd28004850 2023-06-08T03:51:53.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3000e820 con 0x7fbd28004850 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd3effd700 1 -- 172.21.15.79:0/3535448958 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000eb90 con 0x7fbd28004850 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd467fc700 1 -- 172.21.15.79:0/3535448958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 msgr2=0x7fbd28004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd467fc700 1 --2- 172.21.15.79:0/3535448958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd28004cc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009b60 tx=0x7fbd28005880 comp rx=0 tx=0).stop 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd44d99700 1 -- 172.21.15.79:0/3535448958 reap_dead start 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd467fc700 1 -- 172.21.15.79:0/3535448958 shutdown_connections 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd467fc700 1 -- 172.21.15.79:0/3535448958 >> 172.21.15.79:0/3535448958 conn(0x7fbd400cbfe0 msgr2=0x7fbd400cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd467fc700 1 -- 172.21.15.79:0/3535448958 shutdown_connections 2023-06-08T03:51:53.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.496+0000 7fbd467fc700 1 -- 172.21.15.79:0/3535448958 wait complete. 2023-06-08T03:51:53.506 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.498+0000 7fbd467fc700 1 Processor -- start 2023-06-08T03:51:53.507 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.498+0000 7fbd467fc700 1 -- start start 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd401505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd400b4000 0x7fbd400b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd400b4990 0x7fbd400ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd400b5220 con 0x7fbd28004850 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd400b4f40 con 0x7fbd400b4990 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4015eca0 con 0x7fbd400b4000 2023-06-08T03:51:53.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd401505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd401505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37648/0 (socket says 172.21.15.79:37648) 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 -- 172.21.15.79:0/1603887964 learned_addr learned my addr 172.21.15.79:0/1603887964 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 -- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd400b4000 msgr2=0x7fbd400b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd400b4000 0x7fbd400b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 -- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd400b4990 msgr2=0x7fbd400ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd400b4990 0x7fbd400ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 -- 172.21.15.79:0/1603887964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4015ef80 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3ffff700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd401505c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbd340072c0 tx=0x7fbd34007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340077a0 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400f7d0 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4015f260 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400fb80 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.499+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4015f780 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.500+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4015f4a0 con 0x7fbd28004850 2023-06-08T03:51:53.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.501+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd3400fce0 con 0x7fbd28004850 2023-06-08T03:51:53.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.501+0000 7fbd3cff9700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd2c06ace0 0x7fbd2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.501+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbd340894a0 con 0x7fbd28004850 2023-06-08T03:51:53.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.501+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd2c06ace0 0x7fbd2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.502+0000 7fbd3f7fe700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd2c06ace0 0x7fbd2c06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbd30000c00 tx=0x7fbd30013000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.511 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.505+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbd4015f4a0 con 0x7fbd28004850 2023-06-08T03:51:53.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.688+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 --> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd4006a440 con 0x7fbd2c06ace0 2023-06-08T03:51:53.701 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.697+0000 7fbd3cff9700 1 -- 172.21.15.79:0/1603887964 <== mgr.4099 v2:172.21.15.156:6832/2537078349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296070 (secure 0 0 0) 0x7fbd3000fa20 con 0x7fbd2c06ace0 2023-06-08T03:51:53.704 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:53.705 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd2c06ace0 msgr2=0x7fbd2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd2c06ace0 0x7fbd2c06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbd30000c00 tx=0x7fbd30013000 comp rx=0 tx=0).stop 2023-06-08T03:51:53.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 msgr2=0x7fbd401505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.706 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd401505c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbd340072c0 tx=0x7fbd34007680 comp rx=0 tx=0).stop 2023-06-08T03:51:53.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 shutdown_connections 2023-06-08T03:51:53.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd400b4990 0x7fbd400ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd28004850 0x7fbd401505c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.707 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbd2c06ace0 0x7fbd2c06d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 --2- 172.21.15.79:0/1603887964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd400b4000 0x7fbd400b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 >> 172.21.15.79:0/1603887964 conn(0x7fbd400cbfe0 msgr2=0x7fbd400aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:53.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 shutdown_connections 2023-06-08T03:51:53.708 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.700+0000 7fbd467fc700 1 -- 172.21.15.79:0/1603887964 wait complete. 2023-06-08T03:51:53.708 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-08T03:51:53.716 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-06-08T03:51:51.893739+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219816,"kb_used_data":4200,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754904,"statfs":{"total":773094113280,"available":772869021696,"internally_reserved":0,"allocated":4300800,"data_stored":2507880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000849"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.587071+0000","last_change":"2023-06-08T03:50:19.804887+0000","last_active":"2023-06-08T03:51:43.587071+0000","last_peered":"2023-06-08T03:51:43.587071+0000","last_clean":"2023-06-08T03:51:43.587071+0000","last_became_active":"2023-06-08T03:50:19.804546+0000","last_became_peered":"2023-06-08T03:50:19.804546+0000","last_unstale":"2023-06-08T03:51:43.587071+0000","last_undegraded":"2023-06-08T03:51:43.587071+0000","last_fullsized":"2023-06-08T03:51:43.587071+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:35.884773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.400885+0000","last_change":"2023-06-08T03:50:19.172271+0000","last_active":"2023-06-08T03:51:43.400885+0000","last_peered":"2023-06-08T03:51:43.400885+0000","last_clean":"2023-06-08T03:51:43.400885+0000","last_became_active":"2023-06-08T03:50:19.171917+0000","last_became_peered":"2023-06-08T03:50:19.171917+0000","last_unstale":"2023-06-08T03:51:43.400885+0000","last_undegraded":"2023-06-08T03:51:43.400885+0000","last_fullsized":"2023-06-08T03:51:43.400885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:50:46.319373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.600041+0000","last_change":"2023-06-08T03:50:19.171073+0000","last_active":"2023-06-08T03:51:43.600041+0000","last_peered":"2023-06-08T03:51:43.600041+0000","last_clean":"2023-06-08T03:51:43.600041+0000","last_became_active":"2023-06-08T03:50:19.170775+0000","last_became_peered":"2023-06-08T03:50:19.170775+0000","last_unstale":"2023-06-08T03:51:43.600041+0000","last_undegraded":"2023-06-08T03:51:43.600041+0000","last_fullsized":"2023-06-08T03:51:43.600041+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:59:20.969750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":187,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.399646+0000","last_change":"2023-06-08T03:50:19.167800+0000","last_active":"2023-06-08T03:51:43.399646+0000","last_peered":"2023-06-08T03:51:43.399646+0000","last_clean":"2023-06-08T03:51:43.399646+0000","last_became_active":"2023-06-08T03:50:19.167412+0000","last_became_peered":"2023-06-08T03:50:19.167412+0000","last_unstale":"2023-06-08T03:51:43.399646+0000","last_undegraded":"2023-06-08T03:51:43.399646+0000","last_fullsized":"2023-06-08T03:51:43.399646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:45:15.679821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.599553+0000","last_change":"2023-06-08T03:50:19.171834+0000","last_active":"2023-06-08T03:51:43.599553+0000","last_peered":"2023-06-08T03:51:43.599553+0000","last_clean":"2023-06-08T03:51:43.599553+0000","last_became_active":"2023-06-08T03:50:19.171541+0000","last_became_peered":"2023-06-08T03:50:19.171541+0000","last_unstale":"2023-06-08T03:51:43.599553+0000","last_undegraded":"2023-06-08T03:51:43.599553+0000","last_fullsized":"2023-06-08T03:51:43.599553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:34:01.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.599540+0000","last_change":"2023-06-08T03:50:19.170601+0000","last_active":"2023-06-08T03:51:43.599540+0000","last_peered":"2023-06-08T03:51:43.599540+0000","last_clean":"2023-06-08T03:51:43.599540+0000","last_became_active":"2023-06-08T03:50:19.170344+0000","last_became_peered":"2023-06-08T03:50:19.170344+0000","last_unstale":"2023-06-08T03:51:43.599540+0000","last_undegraded":"2023-06-08T03:51:43.599540+0000","last_fullsized":"2023-06-08T03:51:43.599540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:47:09.016262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.399646+0000","last_change":"2023-06-08T03:50:19.805195+0000","last_active":"2023-06-08T03:51:43.399646+0000","last_peered":"2023-06-08T03:51:43.399646+0000","last_clean":"2023-06-08T03:51:43.399646+0000","last_became_active":"2023-06-08T03:50:19.804708+0000","last_became_peered":"2023-06-08T03:50:19.804708+0000","last_unstale":"2023-06-08T03:51:43.399646+0000","last_undegraded":"2023-06-08T03:51:43.399646+0000","last_fullsized":"2023-06-08T03:51:43.399646+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:40:10.638740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.399649+0000","last_change":"2023-06-08T03:50:19.805012+0000","last_active":"2023-06-08T03:51:43.399649+0000","last_peered":"2023-06-08T03:51:43.399649+0000","last_clean":"2023-06-08T03:51:43.399649+0000","last_became_active":"2023-06-08T03:50:19.804722+0000","last_became_peered":"2023-06-08T03:50:19.804722+0000","last_unstale":"2023-06-08T03:51:43.399649+0000","last_undegraded":"2023-06-08T03:51:43.399649+0000","last_fullsized":"2023-06-08T03:51:43.399649+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:25:47.220701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:49.225956+0000","last_change":"2023-06-08T03:51:49.225956+0000","last_active":"2023-06-08T03:51:49.225956+0000","last_peered":"2023-06-08T03:51:49.225956+0000","last_clean":"2023-06-08T03:51:49.225956+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:49.225956+0000","last_undegraded":"2023-06-08T03:51:49.225956+0000","last_fullsized":"2023-06-08T03:51:49.225956+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-06-08T03:51:48.221873+0000","last_change":"2023-06-08T03:51:48.221873+0000","last_active":"2023-06-08T03:51:48.221873+0000","last_peered":"2023-06-08T03:51:48.221873+0000","last_clean":"2023-06-08T03:51:48.221873+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-06-08T03:51:48.221873+0000","last_undegraded":"2023-06-08T03:51:48.221873+0000","last_fullsized":"2023-06-08T03:51:48.221873+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":43,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-06-08T03:51:43.599540+0000","last_change":"2023-06-08T03:50:19.805107+0000","last_active":"2023-06-08T03:51:43.599540+0000","last_peered":"2023-06-08T03:51:43.599540+0000","last_clean":"2023-06-08T03:51:43.599540+0000","last_became_active":"2023-06-08T03:50:19.804754+0000","last_became_peered":"2023-06-08T03:50:19.804754+0000","last_unstale":"2023-06-08T03:51:43.599540+0000","last_undegraded":"2023-06-08T03:51:43.599540+0000","last_fullsized":"2023-06-08T03:51:43.599540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:42:34.460421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":165893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.465999999999999,"5min":11.465999999999999,"15min":11.465999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.9119999999999999,"5min":3.9119999999999999,"15min":3.9119999999999999},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.441},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.505}]},{"osd":6,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":165893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.1230000000000002,"5min":6.1230000000000002,"15min":6.1230000000000002},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.442},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":165893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":21.992000000000001,"5min":21.992000000000001,"15min":21.992000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.25}]},{"osd":2,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":5.4109999999999996,"5min":5.4109999999999996,"15min":5.4109999999999996},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.2690000000000001,"5min":5.2690000000000001,"15min":5.2690000000000001},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":15.984999999999999,"5min":15.984999999999999,"15min":15.984999999999999},"last":0.37},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":18.992000000000001,"5min":18.992000000000001,"15min":18.992000000000001},"last":0.307}]},{"osd":4,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.62},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":15.955,"5min":15.955,"15min":15.955},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":10.962999999999999,"5min":10.962999999999999,"15min":10.962999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":11.023,"5min":11.023,"15min":11.023},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":10.928000000000001,"5min":10.928000000000001,"15min":10.928000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":11.051,"5min":11.051,"15min":11.051},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.923,"5min":15.923,"15min":15.923},"last":0.439},{"interface":"front","average":{"1min":1.696,"5min":1.696,"15min":1.696},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":18.943999999999999,"5min":18.943999999999999,"15min":18.943999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":165893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.222},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.26700000000000002}]},{"osd":2,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.253},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.23799999999999999}]},{"osd":3,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.27800000000000002}]},{"osd":4,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.44700000000000001}]},{"osd":6,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.36799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":165893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":0.315}]},{"osd":1,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":5.3929999999999998,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":0.373}]},{"osd":3,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.5149999999999997,"5min":6.5149999999999997,"15min":6.5149999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.3630000000000004,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.94299999999999995}]},{"osd":6,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.4589999999999996,"5min":6.4589999999999996,"15min":6.4589999999999996},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.78900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":756261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.316},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.36199999999999999}]},{"osd":4,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42099999999999999}]},{"osd":5,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.498},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.58699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":389120,"data_stored":165893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.79900000000000004}]},{"osd":1,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.85899999999999999}]},{"osd":3,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.627}]},{"osd":6,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.83899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":983040,"data_stored":756261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.48399999999999999}]},{"osd":1,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.505}]},{"osd":2,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.442},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.59599999999999997}]},{"osd":4,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-08T03:51:53.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-08T03:51:53.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.872+0000 7f11da287700 1 Processor -- start 2023-06-08T03:51:53.876 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.872+0000 7f11da287700 1 -- start start 2023-06-08T03:51:53.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11da287700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b13b0 0x7f11d40cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11da287700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11d415e250 0x7f11d415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.878 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11da287700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d406bb90 con 0x7f11d40b0fe0 2023-06-08T03:51:53.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11da287700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d415de00 con 0x7f11d415e250 2023-06-08T03:51:53.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11da287700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40b5080 con 0x7f11d40b13b0 2023-06-08T03:51:53.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d8824700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f11d415e250 0x7f11d415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d3fff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f11d40b0fe0 0x7f11d415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32900/0 (socket says 172.21.15.79:32900) 2023-06-08T03:51:53.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d3fff700 1 -- 172.21.15.79:0/4202519100 learned_addr learned my addr 172.21.15.79:0/4202519100 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:53.879 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d37fe700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b13b0 0x7f11d40cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d8824700 1 -- 172.21.15.79:0/4202519100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b13b0 msgr2=0x7f11d40cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d8824700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b13b0 0x7f11d40cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d8824700 1 -- 172.21.15.79:0/4202519100 >> v1:172.21.15.79:6789/0 conn(0x7f11d40b0fe0 legacy=0x7f11d415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d2ffd700 1 -- 172.21.15.79:0/4202519100 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592834178 0 0) 0x7f11d406bb90 con 0x7f11d40b0fe0 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d8824700 1 -- 172.21.15.79:0/4202519100 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d406a1a0 con 0x7f11d415e250 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d37fe700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b13b0 0x7f11d40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:53.880 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.873+0000 7f11d8824700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d415e250 0x7f11d415e6d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009d90 tx=0x7f11cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1c64ebdddc5ade server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11d2ffd700 1 -- 172.21.15.79:0/4202519100 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00e070 con 0x7f11d415e250 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11d2ffd700 1 -- 172.21.15.79:0/4202519100 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc00bbd0 con 0x7f11d415e250 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11d2ffd700 1 -- 172.21.15.79:0/4202519100 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc012430 con 0x7f11d415e250 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 -- 172.21.15.79:0/4202519100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d415e250 msgr2=0x7f11d415e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d415e250 0x7f11d415e6d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009d90 tx=0x7f11cc00b330 comp rx=0 tx=0).stop 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 -- 172.21.15.79:0/4202519100 shutdown_connections 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d415e250 0x7f11d415e6d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.881 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 --2- 172.21.15.79:0/4202519100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b13b0 0x7f11d40cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 -- 172.21.15.79:0/4202519100 >> 172.21.15.79:0/4202519100 conn(0x7f11d40cbfe0 msgr2=0x7f11d40cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:53.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 -- 172.21.15.79:0/4202519100 shutdown_connections 2023-06-08T03:51:53.882 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.874+0000 7f11da287700 1 -- 172.21.15.79:0/4202519100 wait complete. 2023-06-08T03:51:53.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.875+0000 7f11da287700 1 Processor -- start 2023-06-08T03:51:53.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 -- start start 2023-06-08T03:51:53.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b0fe0 0x7f11d4159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 0x7f11d40c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.884 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d415e250 0x7f11d415c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40c56b0 con 0x7f11d40b13b0 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40c5ad0 con 0x7f11d415e250 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11da287700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d415a270 con 0x7f11d40b0fe0 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 0x7f11d40c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11d37fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 0x7f11d40c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37664/0 (socket says 172.21.15.79:37664) 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.876+0000 7f11d37fe700 1 -- 172.21.15.79:0/589747372 learned_addr learned my addr 172.21.15.79:0/589747372 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:53.885 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d3fff700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b0fe0 0x7f11d4159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d37fe700 1 -- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b0fe0 msgr2=0x7f11d4159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d37fe700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b0fe0 0x7f11d4159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d37fe700 1 -- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d415e250 msgr2=0x7f11d415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d8824700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d415e250 0x7f11d415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d37fe700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d415e250 0x7f11d415c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d37fe700 1 -- 172.21.15.79:0/589747372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc009a70 con 0x7f11d40b13b0 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d37fe700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 0x7f11d40c40c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c750 tx=0x7f11c400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.886 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4011800 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4011960 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d40c4a30 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4011cd0 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.877+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d40c4ff0 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.878+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d40c57f0 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.878+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11c4021350 con 0x7f11d40b13b0 2023-06-08T03:51:53.887 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.879+0000 7f11d0ff9700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f11c006ace0 0x7f11c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:53.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.879+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f11c40895c0 con 0x7f11d40b13b0 2023-06-08T03:51:53.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.879+0000 7f11d3fff700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f11c006ace0 0x7f11c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:53.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.880+0000 7f11d3fff700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f11c006ace0 0x7f11c006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000f60 tx=0x7f11c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:53.888 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:53.883+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11d40c57f0 con 0x7f11d40b13b0 2023-06-08T03:51:54.074 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.070+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 --> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11d4071eb0 con 0x7f11c006ace0 2023-06-08T03:51:54.083 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.079+0000 7f11d0ff9700 1 -- 172.21.15.79:0/589747372 <== mgr.4099 v2:172.21.15.156:6832/2537078349 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310869 (secure 0 0 0) 0x7f11c80070d0 con 0x7f11c006ace0 2023-06-08T03:51:54.086 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:54.087 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f11c006ace0 msgr2=0x7f11c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f11c006ace0 0x7f11c006d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000f60 tx=0x7f11c8009040 comp rx=0 tx=0).stop 2023-06-08T03:51:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 msgr2=0x7f11d40c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 0x7f11d40c40c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c750 tx=0x7f11c400cb10 comp rx=0 tx=0).stop 2023-06-08T03:51:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 shutdown_connections 2023-06-08T03:51:54.088 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d415e250 0x7f11d415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f11c006ace0 0x7f11c006d190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d40b0fe0 0x7f11d4159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.082+0000 7f11da287700 1 --2- 172.21.15.79:0/589747372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f11d40b13b0 0x7f11d40c40c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.083+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 >> 172.21.15.79:0/589747372 conn(0x7f11d40cbfe0 msgr2=0x7f11d40b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.083+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 shutdown_connections 2023-06-08T03:51:54.089 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.083+0000 7f11da287700 1 -- 172.21.15.79:0/589747372 wait complete. 2023-06-08T03:51:54.090 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2023-06-08T03:51:54.098 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-06-08T03:51:53.894134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":195,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220252,"kb_used_data":4636,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754468,"statfs":{"total":773094113280,"available":772868575232,"internally_reserved":0,"allocated":4747264,"data_stored":2728503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.666990"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402373+0000","last_change":"2023-06-08T03:51:49.658872+0000","last_active":"2023-06-08T03:51:51.402373+0000","last_peered":"2023-06-08T03:51:51.402373+0000","last_clean":"2023-06-08T03:51:51.402373+0000","last_became_active":"2023-06-08T03:51:49.658625+0000","last_became_peered":"2023-06-08T03:51:49.658625+0000","last_unstale":"2023-06-08T03:51:51.402373+0000","last_undegraded":"2023-06-08T03:51:51.402373+0000","last_fullsized":"2023-06-08T03:51:51.402373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:35:43.430143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402412+0000","last_change":"2023-06-08T03:51:49.659472+0000","last_active":"2023-06-08T03:51:51.402412+0000","last_peered":"2023-06-08T03:51:51.402412+0000","last_clean":"2023-06-08T03:51:51.402412+0000","last_became_active":"2023-06-08T03:51:49.659158+0000","last_became_peered":"2023-06-08T03:51:49.659158+0000","last_unstale":"2023-06-08T03:51:51.402412+0000","last_undegraded":"2023-06-08T03:51:51.402412+0000","last_fullsized":"2023-06-08T03:51:51.402412+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:57:40.918193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402654+0000","last_change":"2023-06-08T03:51:49.659287+0000","last_active":"2023-06-08T03:51:51.402654+0000","last_peered":"2023-06-08T03:51:51.402654+0000","last_clean":"2023-06-08T03:51:51.402654+0000","last_became_active":"2023-06-08T03:51:49.659043+0000","last_became_peered":"2023-06-08T03:51:49.659043+0000","last_unstale":"2023-06-08T03:51:51.402654+0000","last_undegraded":"2023-06-08T03:51:51.402654+0000","last_fullsized":"2023-06-08T03:51:51.402654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:30:14.498669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402445+0000","last_change":"2023-06-08T03:51:49.660756+0000","last_active":"2023-06-08T03:51:51.402445+0000","last_peered":"2023-06-08T03:51:51.402445+0000","last_clean":"2023-06-08T03:51:51.402445+0000","last_became_active":"2023-06-08T03:51:49.660470+0000","last_became_peered":"2023-06-08T03:51:49.660470+0000","last_unstale":"2023-06-08T03:51:51.402445+0000","last_undegraded":"2023-06-08T03:51:51.402445+0000","last_fullsized":"2023-06-08T03:51:51.402445+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:50.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.401707+0000","last_change":"2023-06-08T03:51:49.649043+0000","last_active":"2023-06-08T03:51:51.401707+0000","last_peered":"2023-06-08T03:51:51.401707+0000","last_clean":"2023-06-08T03:51:51.401707+0000","last_became_active":"2023-06-08T03:51:49.647582+0000","last_became_peered":"2023-06-08T03:51:49.647582+0000","last_unstale":"2023-06-08T03:51:51.401707+0000","last_undegraded":"2023-06-08T03:51:51.401707+0000","last_fullsized":"2023-06-08T03:51:51.401707+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:33:48.742320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640224+0000","last_change":"2023-06-08T03:51:49.655851+0000","last_active":"2023-06-08T03:51:50.640224+0000","last_peered":"2023-06-08T03:51:50.640224+0000","last_clean":"2023-06-08T03:51:50.640224+0000","last_became_active":"2023-06-08T03:51:49.655545+0000","last_became_peered":"2023-06-08T03:51:49.655545+0000","last_unstale":"2023-06-08T03:51:50.640224+0000","last_undegraded":"2023-06-08T03:51:50.640224+0000","last_fullsized":"2023-06-08T03:51:50.640224+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:31:40.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638586+0000","last_change":"2023-06-08T03:51:49.662659+0000","last_active":"2023-06-08T03:51:50.638586+0000","last_peered":"2023-06-08T03:51:50.638586+0000","last_clean":"2023-06-08T03:51:50.638586+0000","last_became_active":"2023-06-08T03:51:49.662380+0000","last_became_peered":"2023-06-08T03:51:49.662380+0000","last_unstale":"2023-06-08T03:51:50.638586+0000","last_undegraded":"2023-06-08T03:51:50.638586+0000","last_fullsized":"2023-06-08T03:51:50.638586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:29:27.420989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402057+0000","last_change":"2023-06-08T03:51:49.648101+0000","last_active":"2023-06-08T03:51:51.402057+0000","last_peered":"2023-06-08T03:51:51.402057+0000","last_clean":"2023-06-08T03:51:51.402057+0000","last_became_active":"2023-06-08T03:51:49.647792+0000","last_became_peered":"2023-06-08T03:51:49.647792+0000","last_unstale":"2023-06-08T03:51:51.402057+0000","last_undegraded":"2023-06-08T03:51:51.402057+0000","last_fullsized":"2023-06-08T03:51:51.402057+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:03:12.222645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640155+0000","last_change":"2023-06-08T03:51:49.655089+0000","last_active":"2023-06-08T03:51:50.640155+0000","last_peered":"2023-06-08T03:51:50.640155+0000","last_clean":"2023-06-08T03:51:50.640155+0000","last_became_active":"2023-06-08T03:51:49.654796+0000","last_became_peered":"2023-06-08T03:51:49.654796+0000","last_unstale":"2023-06-08T03:51:50.640155+0000","last_undegraded":"2023-06-08T03:51:50.640155+0000","last_fullsized":"2023-06-08T03:51:50.640155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:27:40.058651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083691+0000","last_change":"2023-06-08T03:51:49.659616+0000","last_active":"2023-06-08T03:51:51.083691+0000","last_peered":"2023-06-08T03:51:51.083691+0000","last_clean":"2023-06-08T03:51:51.083691+0000","last_became_active":"2023-06-08T03:51:49.659276+0000","last_became_peered":"2023-06-08T03:51:49.659276+0000","last_unstale":"2023-06-08T03:51:51.083691+0000","last_undegraded":"2023-06-08T03:51:51.083691+0000","last_fullsized":"2023-06-08T03:51:51.083691+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:43:50.227851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402650+0000","last_change":"2023-06-08T03:51:49.649626+0000","last_active":"2023-06-08T03:51:51.402650+0000","last_peered":"2023-06-08T03:51:51.402650+0000","last_clean":"2023-06-08T03:51:51.402650+0000","last_became_active":"2023-06-08T03:51:49.649296+0000","last_became_peered":"2023-06-08T03:51:49.649296+0000","last_unstale":"2023-06-08T03:51:51.402650+0000","last_undegraded":"2023-06-08T03:51:51.402650+0000","last_fullsized":"2023-06-08T03:51:51.402650+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:45:07.842972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.401919+0000","last_change":"2023-06-08T03:51:49.660596+0000","last_active":"2023-06-08T03:51:51.401919+0000","last_peered":"2023-06-08T03:51:51.401919+0000","last_clean":"2023-06-08T03:51:51.401919+0000","last_became_active":"2023-06-08T03:51:49.659838+0000","last_became_peered":"2023-06-08T03:51:49.659838+0000","last_unstale":"2023-06-08T03:51:51.401919+0000","last_undegraded":"2023-06-08T03:51:51.401919+0000","last_fullsized":"2023-06-08T03:51:51.401919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:48:40.089700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640255+0000","last_change":"2023-06-08T03:51:49.655867+0000","last_active":"2023-06-08T03:51:50.640255+0000","last_peered":"2023-06-08T03:51:50.640255+0000","last_clean":"2023-06-08T03:51:50.640255+0000","last_became_active":"2023-06-08T03:51:49.655577+0000","last_became_peered":"2023-06-08T03:51:49.655577+0000","last_unstale":"2023-06-08T03:51:50.640255+0000","last_undegraded":"2023-06-08T03:51:50.640255+0000","last_fullsized":"2023-06-08T03:51:50.640255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:18:06.958212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402209+0000","last_change":"2023-06-08T03:51:49.665801+0000","last_active":"2023-06-08T03:51:51.402209+0000","last_peered":"2023-06-08T03:51:51.402209+0000","last_clean":"2023-06-08T03:51:51.402209+0000","last_became_active":"2023-06-08T03:51:49.665515+0000","last_became_peered":"2023-06-08T03:51:49.665515+0000","last_unstale":"2023-06-08T03:51:51.402209+0000","last_undegraded":"2023-06-08T03:51:51.402209+0000","last_fullsized":"2023-06-08T03:51:51.402209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:46:28.715502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402093+0000","last_change":"2023-06-08T03:51:49.652640+0000","last_active":"2023-06-08T03:51:51.402093+0000","last_peered":"2023-06-08T03:51:51.402093+0000","last_clean":"2023-06-08T03:51:51.402093+0000","last_became_active":"2023-06-08T03:51:49.652377+0000","last_became_peered":"2023-06-08T03:51:49.652377+0000","last_unstale":"2023-06-08T03:51:51.402093+0000","last_undegraded":"2023-06-08T03:51:51.402093+0000","last_fullsized":"2023-06-08T03:51:51.402093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:47:08.440886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402838+0000","last_change":"2023-06-08T03:51:49.658239+0000","last_active":"2023-06-08T03:51:51.402838+0000","last_peered":"2023-06-08T03:51:51.402838+0000","last_clean":"2023-06-08T03:51:51.402838+0000","last_became_active":"2023-06-08T03:51:49.658018+0000","last_became_peered":"2023-06-08T03:51:49.658018+0000","last_unstale":"2023-06-08T03:51:51.402838+0000","last_undegraded":"2023-06-08T03:51:51.402838+0000","last_fullsized":"2023-06-08T03:51:51.402838+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:43:41.899876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402305+0000","last_change":"2023-06-08T03:51:49.655889+0000","last_active":"2023-06-08T03:51:51.402305+0000","last_peered":"2023-06-08T03:51:51.402305+0000","last_clean":"2023-06-08T03:51:51.402305+0000","last_became_active":"2023-06-08T03:51:49.655630+0000","last_became_peered":"2023-06-08T03:51:49.655630+0000","last_unstale":"2023-06-08T03:51:51.402305+0000","last_undegraded":"2023-06-08T03:51:51.402305+0000","last_fullsized":"2023-06-08T03:51:51.402305+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:15:49.612923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402751+0000","last_change":"2023-06-08T03:51:49.656345+0000","last_active":"2023-06-08T03:51:51.402751+0000","last_peered":"2023-06-08T03:51:51.402751+0000","last_clean":"2023-06-08T03:51:51.402751+0000","last_became_active":"2023-06-08T03:51:49.656098+0000","last_became_peered":"2023-06-08T03:51:49.656098+0000","last_unstale":"2023-06-08T03:51:51.402751+0000","last_undegraded":"2023-06-08T03:51:51.402751+0000","last_fullsized":"2023-06-08T03:51:51.402751+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:05:28.621909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402731+0000","last_change":"2023-06-08T03:51:49.647823+0000","last_active":"2023-06-08T03:51:51.402731+0000","last_peered":"2023-06-08T03:51:51.402731+0000","last_clean":"2023-06-08T03:51:51.402731+0000","last_became_active":"2023-06-08T03:51:49.647554+0000","last_became_peered":"2023-06-08T03:51:49.647554+0000","last_unstale":"2023-06-08T03:51:51.402731+0000","last_undegraded":"2023-06-08T03:51:51.402731+0000","last_fullsized":"2023-06-08T03:51:51.402731+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:23:18.727290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083593+0000","last_change":"2023-06-08T03:51:49.655807+0000","last_active":"2023-06-08T03:51:51.083593+0000","last_peered":"2023-06-08T03:51:51.083593+0000","last_clean":"2023-06-08T03:51:51.083593+0000","last_became_active":"2023-06-08T03:51:49.655488+0000","last_became_peered":"2023-06-08T03:51:49.655488+0000","last_unstale":"2023-06-08T03:51:51.083593+0000","last_undegraded":"2023-06-08T03:51:51.083593+0000","last_fullsized":"2023-06-08T03:51:51.083593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:52:24.928753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402283+0000","last_change":"2023-06-08T03:51:49.658484+0000","last_active":"2023-06-08T03:51:51.402283+0000","last_peered":"2023-06-08T03:51:51.402283+0000","last_clean":"2023-06-08T03:51:51.402283+0000","last_became_active":"2023-06-08T03:51:49.658203+0000","last_became_peered":"2023-06-08T03:51:49.658203+0000","last_unstale":"2023-06-08T03:51:51.402283+0000","last_undegraded":"2023-06-08T03:51:51.402283+0000","last_fullsized":"2023-06-08T03:51:51.402283+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:48:40.089700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639436+0000","last_change":"2023-06-08T03:51:49.650919+0000","last_active":"2023-06-08T03:51:50.639436+0000","last_peered":"2023-06-08T03:51:50.639436+0000","last_clean":"2023-06-08T03:51:50.639436+0000","last_became_active":"2023-06-08T03:51:49.650612+0000","last_became_peered":"2023-06-08T03:51:49.650612+0000","last_unstale":"2023-06-08T03:51:50.639436+0000","last_undegraded":"2023-06-08T03:51:50.639436+0000","last_fullsized":"2023-06-08T03:51:50.639436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:29:04.322227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638571+0000","last_change":"2023-06-08T03:51:49.662633+0000","last_active":"2023-06-08T03:51:50.638571+0000","last_peered":"2023-06-08T03:51:50.638571+0000","last_clean":"2023-06-08T03:51:50.638571+0000","last_became_active":"2023-06-08T03:51:49.661297+0000","last_became_peered":"2023-06-08T03:51:49.661297+0000","last_unstale":"2023-06-08T03:51:50.638571+0000","last_undegraded":"2023-06-08T03:51:50.638571+0000","last_fullsized":"2023-06-08T03:51:50.638571+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:37:07.118115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402225+0000","last_change":"2023-06-08T03:51:49.661116+0000","last_active":"2023-06-08T03:51:51.402225+0000","last_peered":"2023-06-08T03:51:51.402225+0000","last_clean":"2023-06-08T03:51:51.402225+0000","last_became_active":"2023-06-08T03:51:49.660841+0000","last_became_peered":"2023-06-08T03:51:49.660841+0000","last_unstale":"2023-06-08T03:51:51.402225+0000","last_undegraded":"2023-06-08T03:51:51.402225+0000","last_fullsized":"2023-06-08T03:51:51.402225+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:03:27.730709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402560+0000","last_change":"2023-06-08T03:51:49.658469+0000","last_active":"2023-06-08T03:51:51.402560+0000","last_peered":"2023-06-08T03:51:51.402560+0000","last_clean":"2023-06-08T03:51:51.402560+0000","last_became_active":"2023-06-08T03:51:49.658166+0000","last_became_peered":"2023-06-08T03:51:49.658166+0000","last_unstale":"2023-06-08T03:51:51.402560+0000","last_undegraded":"2023-06-08T03:51:51.402560+0000","last_fullsized":"2023-06-08T03:51:51.402560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:31:40.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402616+0000","last_change":"2023-06-08T03:51:49.654785+0000","last_active":"2023-06-08T03:51:51.402616+0000","last_peered":"2023-06-08T03:51:51.402616+0000","last_clean":"2023-06-08T03:51:51.402616+0000","last_became_active":"2023-06-08T03:51:49.654528+0000","last_became_peered":"2023-06-08T03:51:49.654528+0000","last_unstale":"2023-06-08T03:51:51.402616+0000","last_undegraded":"2023-06-08T03:51:51.402616+0000","last_fullsized":"2023-06-08T03:51:51.402616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:23:56.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639059+0000","last_change":"2023-06-08T03:51:49.663422+0000","last_active":"2023-06-08T03:51:50.639059+0000","last_peered":"2023-06-08T03:51:50.639059+0000","last_clean":"2023-06-08T03:51:50.639059+0000","last_became_active":"2023-06-08T03:51:49.662856+0000","last_became_peered":"2023-06-08T03:51:49.662856+0000","last_unstale":"2023-06-08T03:51:50.639059+0000","last_undegraded":"2023-06-08T03:51:50.639059+0000","last_fullsized":"2023-06-08T03:51:50.639059+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:57:50.081685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639052+0000","last_change":"2023-06-08T03:51:49.663332+0000","last_active":"2023-06-08T03:51:50.639052+0000","last_peered":"2023-06-08T03:51:50.639052+0000","last_clean":"2023-06-08T03:51:50.639052+0000","last_became_active":"2023-06-08T03:51:49.663047+0000","last_became_peered":"2023-06-08T03:51:49.663047+0000","last_unstale":"2023-06-08T03:51:50.639052+0000","last_undegraded":"2023-06-08T03:51:50.639052+0000","last_fullsized":"2023-06-08T03:51:50.639052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:34:29.658902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.401839+0000","last_change":"2023-06-08T03:51:49.657419+0000","last_active":"2023-06-08T03:51:51.401839+0000","last_peered":"2023-06-08T03:51:51.401839+0000","last_clean":"2023-06-08T03:51:51.401839+0000","last_became_active":"2023-06-08T03:51:49.657147+0000","last_became_peered":"2023-06-08T03:51:49.657147+0000","last_unstale":"2023-06-08T03:51:51.401839+0000","last_undegraded":"2023-06-08T03:51:51.401839+0000","last_fullsized":"2023-06-08T03:51:51.401839+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:34:25.055445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.401770+0000","last_change":"2023-06-08T03:51:49.661217+0000","last_active":"2023-06-08T03:51:51.401770+0000","last_peered":"2023-06-08T03:51:51.401770+0000","last_clean":"2023-06-08T03:51:51.401770+0000","last_became_active":"2023-06-08T03:51:49.660773+0000","last_became_peered":"2023-06-08T03:51:49.660773+0000","last_unstale":"2023-06-08T03:51:51.401770+0000","last_undegraded":"2023-06-08T03:51:51.401770+0000","last_fullsized":"2023-06-08T03:51:51.401770+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:46:28.715502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639436+0000","last_change":"2023-06-08T03:51:49.649426+0000","last_active":"2023-06-08T03:51:50.639436+0000","last_peered":"2023-06-08T03:51:50.639436+0000","last_clean":"2023-06-08T03:51:50.639436+0000","last_became_active":"2023-06-08T03:51:49.649076+0000","last_became_peered":"2023-06-08T03:51:49.649076+0000","last_unstale":"2023-06-08T03:51:50.639436+0000","last_undegraded":"2023-06-08T03:51:50.639436+0000","last_fullsized":"2023-06-08T03:51:50.639436+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:17:58.888485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402018+0000","last_change":"2023-06-08T03:51:49.651797+0000","last_active":"2023-06-08T03:51:51.402018+0000","last_peered":"2023-06-08T03:51:51.402018+0000","last_clean":"2023-06-08T03:51:51.402018+0000","last_became_active":"2023-06-08T03:51:49.651509+0000","last_became_peered":"2023-06-08T03:51:49.651509+0000","last_unstale":"2023-06-08T03:51:51.402018+0000","last_undegraded":"2023-06-08T03:51:51.402018+0000","last_fullsized":"2023-06-08T03:51:51.402018+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:34:34.758896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639665+0000","last_change":"2023-06-08T03:51:49.659268+0000","last_active":"2023-06-08T03:51:50.639665+0000","last_peered":"2023-06-08T03:51:50.639665+0000","last_clean":"2023-06-08T03:51:50.639665+0000","last_became_active":"2023-06-08T03:51:49.658985+0000","last_became_peered":"2023-06-08T03:51:49.658985+0000","last_unstale":"2023-06-08T03:51:50.639665+0000","last_undegraded":"2023-06-08T03:51:50.639665+0000","last_fullsized":"2023-06-08T03:51:50.639665+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:36:52.447372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"37'4","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640104+0000","last_change":"2023-06-08T03:51:49.246757+0000","last_active":"2023-06-08T03:51:50.640104+0000","last_peered":"2023-06-08T03:51:50.640104+0000","last_clean":"2023-06-08T03:51:50.640104+0000","last_became_active":"2023-06-08T03:51:49.246480+0000","last_became_peered":"2023-06-08T03:51:49.246480+0000","last_unstale":"2023-06-08T03:51:50.640104+0000","last_undegraded":"2023-06-08T03:51:50.640104+0000","last_fullsized":"2023-06-08T03:51:50.640104+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:35:42.426060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402422+0000","last_change":"2023-06-08T03:51:49.651963+0000","last_active":"2023-06-08T03:51:51.402422+0000","last_peered":"2023-06-08T03:51:51.402422+0000","last_clean":"2023-06-08T03:51:51.402422+0000","last_became_active":"2023-06-08T03:51:49.651715+0000","last_became_peered":"2023-06-08T03:51:49.651715+0000","last_unstale":"2023-06-08T03:51:51.402422+0000","last_undegraded":"2023-06-08T03:51:51.402422+0000","last_fullsized":"2023-06-08T03:51:51.402422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:24:10.621170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639209+0000","last_change":"2023-06-08T03:51:49.254290+0000","last_active":"2023-06-08T03:51:50.639209+0000","last_peered":"2023-06-08T03:51:50.639209+0000","last_clean":"2023-06-08T03:51:50.639209+0000","last_became_active":"2023-06-08T03:51:49.254034+0000","last_became_peered":"2023-06-08T03:51:49.254034+0000","last_unstale":"2023-06-08T03:51:50.639209+0000","last_undegraded":"2023-06-08T03:51:50.639209+0000","last_fullsized":"2023-06-08T03:51:50.639209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:18:07.894448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402597+0000","last_change":"2023-06-08T03:51:49.660847+0000","last_active":"2023-06-08T03:51:51.402597+0000","last_peered":"2023-06-08T03:51:51.402597+0000","last_clean":"2023-06-08T03:51:51.402597+0000","last_became_active":"2023-06-08T03:51:49.659213+0000","last_became_peered":"2023-06-08T03:51:49.659213+0000","last_unstale":"2023-06-08T03:51:51.402597+0000","last_undegraded":"2023-06-08T03:51:51.402597+0000","last_fullsized":"2023-06-08T03:51:51.402597+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:37:29.923128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639525+0000","last_change":"2023-06-08T03:51:49.249615+0000","last_active":"2023-06-08T03:51:50.639525+0000","last_peered":"2023-06-08T03:51:50.639525+0000","last_clean":"2023-06-08T03:51:50.639525+0000","last_became_active":"2023-06-08T03:51:49.249332+0000","last_became_peered":"2023-06-08T03:51:49.249332+0000","last_unstale":"2023-06-08T03:51:50.639525+0000","last_undegraded":"2023-06-08T03:51:50.639525+0000","last_fullsized":"2023-06-08T03:51:50.639525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:33:47.738237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402653+0000","last_change":"2023-06-08T03:51:49.664191+0000","last_active":"2023-06-08T03:51:51.402653+0000","last_peered":"2023-06-08T03:51:51.402653+0000","last_clean":"2023-06-08T03:51:51.402653+0000","last_became_active":"2023-06-08T03:51:49.663079+0000","last_became_peered":"2023-06-08T03:51:49.663079+0000","last_unstale":"2023-06-08T03:51:51.402653+0000","last_undegraded":"2023-06-08T03:51:51.402653+0000","last_fullsized":"2023-06-08T03:51:51.402653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:03:27.730709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639096+0000","last_change":"2023-06-08T03:51:49.251949+0000","last_active":"2023-06-08T03:51:50.639096+0000","last_peered":"2023-06-08T03:51:50.639096+0000","last_clean":"2023-06-08T03:51:50.639096+0000","last_became_active":"2023-06-08T03:51:49.251671+0000","last_became_peered":"2023-06-08T03:51:49.251671+0000","last_unstale":"2023-06-08T03:51:50.639096+0000","last_undegraded":"2023-06-08T03:51:50.639096+0000","last_fullsized":"2023-06-08T03:51:50.639096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:14:59.991165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638642+0000","last_change":"2023-06-08T03:51:49.658882+0000","last_active":"2023-06-08T03:51:50.638642+0000","last_peered":"2023-06-08T03:51:50.638642+0000","last_clean":"2023-06-08T03:51:50.638642+0000","last_became_active":"2023-06-08T03:51:49.658646+0000","last_became_peered":"2023-06-08T03:51:49.658646+0000","last_unstale":"2023-06-08T03:51:50.638642+0000","last_undegraded":"2023-06-08T03:51:50.638642+0000","last_fullsized":"2023-06-08T03:51:50.638642+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:37:42.820912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640123+0000","last_change":"2023-06-08T03:51:49.251685+0000","last_active":"2023-06-08T03:51:50.640123+0000","last_peered":"2023-06-08T03:51:50.640123+0000","last_clean":"2023-06-08T03:51:50.640123+0000","last_became_active":"2023-06-08T03:51:49.251437+0000","last_became_peered":"2023-06-08T03:51:49.251437+0000","last_unstale":"2023-06-08T03:51:50.640123+0000","last_undegraded":"2023-06-08T03:51:50.640123+0000","last_fullsized":"2023-06-08T03:51:50.640123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:24:09.617087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083494+0000","last_change":"2023-06-08T03:51:49.658254+0000","last_active":"2023-06-08T03:51:51.083494+0000","last_peered":"2023-06-08T03:51:51.083494+0000","last_clean":"2023-06-08T03:51:51.083494+0000","last_became_active":"2023-06-08T03:51:49.657948+0000","last_became_peered":"2023-06-08T03:51:49.657948+0000","last_unstale":"2023-06-08T03:51:51.083494+0000","last_undegraded":"2023-06-08T03:51:51.083494+0000","last_fullsized":"2023-06-08T03:51:51.083494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:13:41.646963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638885+0000","last_change":"2023-06-08T03:51:49.251906+0000","last_active":"2023-06-08T03:51:50.638885+0000","last_peered":"2023-06-08T03:51:50.638885+0000","last_clean":"2023-06-08T03:51:50.638885+0000","last_became_active":"2023-06-08T03:51:49.251622+0000","last_became_peered":"2023-06-08T03:51:49.251622+0000","last_unstale":"2023-06-08T03:51:50.638885+0000","last_undegraded":"2023-06-08T03:51:50.638885+0000","last_fullsized":"2023-06-08T03:51:50.638885+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:45:55.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083212+0000","last_change":"2023-06-08T03:51:49.661135+0000","last_active":"2023-06-08T03:51:51.083212+0000","last_peered":"2023-06-08T03:51:51.083212+0000","last_clean":"2023-06-08T03:51:51.083212+0000","last_became_active":"2023-06-08T03:51:49.660860+0000","last_became_peered":"2023-06-08T03:51:49.660860+0000","last_unstale":"2023-06-08T03:51:51.083212+0000","last_undegraded":"2023-06-08T03:51:51.083212+0000","last_fullsized":"2023-06-08T03:51:51.083212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:11:40.052631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.401976+0000","last_change":"2023-06-08T03:51:49.247701+0000","last_active":"2023-06-08T03:51:51.401976+0000","last_peered":"2023-06-08T03:51:51.401976+0000","last_clean":"2023-06-08T03:51:51.401976+0000","last_became_active":"2023-06-08T03:51:49.247390+0000","last_became_peered":"2023-06-08T03:51:49.247390+0000","last_unstale":"2023-06-08T03:51:51.401976+0000","last_undegraded":"2023-06-08T03:51:51.401976+0000","last_fullsized":"2023-06-08T03:51:51.401976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:48:47.328921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638685+0000","last_change":"2023-06-08T03:51:49.650808+0000","last_active":"2023-06-08T03:51:50.638685+0000","last_peered":"2023-06-08T03:51:50.638685+0000","last_clean":"2023-06-08T03:51:50.638685+0000","last_became_active":"2023-06-08T03:51:49.650556+0000","last_became_peered":"2023-06-08T03:51:49.650556+0000","last_unstale":"2023-06-08T03:51:50.638685+0000","last_undegraded":"2023-06-08T03:51:50.638685+0000","last_fullsized":"2023-06-08T03:51:50.638685+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:44:39.736152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083402+0000","last_change":"2023-06-08T03:51:49.248300+0000","last_active":"2023-06-08T03:51:51.083402+0000","last_peered":"2023-06-08T03:51:51.083402+0000","last_clean":"2023-06-08T03:51:51.083402+0000","last_became_active":"2023-06-08T03:51:49.248115+0000","last_became_peered":"2023-06-08T03:51:49.248115+0000","last_unstale":"2023-06-08T03:51:51.083402+0000","last_undegraded":"2023-06-08T03:51:51.083402+0000","last_fullsized":"2023-06-08T03:51:51.083402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:24:08.823944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402835+0000","last_change":"2023-06-08T03:51:49.653310+0000","last_active":"2023-06-08T03:51:51.402835+0000","last_peered":"2023-06-08T03:51:51.402835+0000","last_clean":"2023-06-08T03:51:51.402835+0000","last_became_active":"2023-06-08T03:51:49.653054+0000","last_became_peered":"2023-06-08T03:51:49.653054+0000","last_unstale":"2023-06-08T03:51:51.402835+0000","last_undegraded":"2023-06-08T03:51:51.402835+0000","last_fullsized":"2023-06-08T03:51:51.402835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T08:11:20.314146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402989+0000","last_change":"2023-06-08T03:51:49.241628+0000","last_active":"2023-06-08T03:51:51.402989+0000","last_peered":"2023-06-08T03:51:51.402989+0000","last_clean":"2023-06-08T03:51:51.402989+0000","last_became_active":"2023-06-08T03:51:49.241390+0000","last_became_peered":"2023-06-08T03:51:49.241390+0000","last_unstale":"2023-06-08T03:51:51.402989+0000","last_undegraded":"2023-06-08T03:51:51.402989+0000","last_fullsized":"2023-06-08T03:51:51.402989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:32:04.187215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402632+0000","last_change":"2023-06-08T03:51:49.647689+0000","last_active":"2023-06-08T03:51:51.402632+0000","last_peered":"2023-06-08T03:51:51.402632+0000","last_clean":"2023-06-08T03:51:51.402632+0000","last_became_active":"2023-06-08T03:51:49.647436+0000","last_became_peered":"2023-06-08T03:51:49.647436+0000","last_unstale":"2023-06-08T03:51:51.402632+0000","last_undegraded":"2023-06-08T03:51:51.402632+0000","last_fullsized":"2023-06-08T03:51:51.402632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:12:10.004893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083759+0000","last_change":"2023-06-08T03:51:49.243323+0000","last_active":"2023-06-08T03:51:51.083759+0000","last_peered":"2023-06-08T03:51:51.083759+0000","last_clean":"2023-06-08T03:51:51.083759+0000","last_became_active":"2023-06-08T03:51:49.243013+0000","last_became_peered":"2023-06-08T03:51:49.243013+0000","last_unstale":"2023-06-08T03:51:51.083759+0000","last_undegraded":"2023-06-08T03:51:51.083759+0000","last_fullsized":"2023-06-08T03:51:51.083759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:00:01.836655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402542+0000","last_change":"2023-06-08T03:51:49.655650+0000","last_active":"2023-06-08T03:51:51.402542+0000","last_peered":"2023-06-08T03:51:51.402542+0000","last_clean":"2023-06-08T03:51:51.402542+0000","last_became_active":"2023-06-08T03:51:49.655365+0000","last_became_peered":"2023-06-08T03:51:49.655365+0000","last_unstale":"2023-06-08T03:51:51.402542+0000","last_undegraded":"2023-06-08T03:51:51.402542+0000","last_fullsized":"2023-06-08T03:51:51.402542+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:24:30.922396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"37'3","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.401952+0000","last_change":"2023-06-08T03:51:49.245467+0000","last_active":"2023-06-08T03:51:51.401952+0000","last_peered":"2023-06-08T03:51:51.401952+0000","last_clean":"2023-06-08T03:51:51.401952+0000","last_became_active":"2023-06-08T03:51:49.245179+0000","last_became_peered":"2023-06-08T03:51:49.245179+0000","last_unstale":"2023-06-08T03:51:51.401952+0000","last_undegraded":"2023-06-08T03:51:51.401952+0000","last_fullsized":"2023-06-08T03:51:51.401952+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:44:04.529777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639144+0000","last_change":"2023-06-08T03:51:49.661843+0000","last_active":"2023-06-08T03:51:50.639144+0000","last_peered":"2023-06-08T03:51:50.639144+0000","last_clean":"2023-06-08T03:51:50.639144+0000","last_became_active":"2023-06-08T03:51:49.660342+0000","last_became_peered":"2023-06-08T03:51:49.660342+0000","last_unstale":"2023-06-08T03:51:50.639144+0000","last_undegraded":"2023-06-08T03:51:50.639144+0000","last_fullsized":"2023-06-08T03:51:50.639144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:36:59.575639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"37'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083877+0000","last_change":"2023-06-08T03:51:49.254428+0000","last_active":"2023-06-08T03:51:51.083877+0000","last_peered":"2023-06-08T03:51:51.083877+0000","last_clean":"2023-06-08T03:51:51.083877+0000","last_became_active":"2023-06-08T03:51:49.254180+0000","last_became_peered":"2023-06-08T03:51:49.254180+0000","last_unstale":"2023-06-08T03:51:51.083877+0000","last_undegraded":"2023-06-08T03:51:51.083877+0000","last_fullsized":"2023-06-08T03:51:51.083877+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T09:40:21.445081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402255+0000","last_change":"2023-06-08T03:51:49.655750+0000","last_active":"2023-06-08T03:51:51.402255+0000","last_peered":"2023-06-08T03:51:51.402255+0000","last_clean":"2023-06-08T03:51:51.402255+0000","last_became_active":"2023-06-08T03:51:49.655425+0000","last_became_peered":"2023-06-08T03:51:49.655425+0000","last_unstale":"2023-06-08T03:51:51.402255+0000","last_undegraded":"2023-06-08T03:51:51.402255+0000","last_fullsized":"2023-06-08T03:51:51.402255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:59:27.137941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639738+0000","last_change":"2023-06-08T03:51:49.247546+0000","last_active":"2023-06-08T03:51:50.639738+0000","last_peered":"2023-06-08T03:51:50.639738+0000","last_clean":"2023-06-08T03:51:50.639738+0000","last_became_active":"2023-06-08T03:51:49.247200+0000","last_became_peered":"2023-06-08T03:51:49.247200+0000","last_unstale":"2023-06-08T03:51:50.639738+0000","last_undegraded":"2023-06-08T03:51:50.639738+0000","last_fullsized":"2023-06-08T03:51:50.639738+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:41:06.615472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639581+0000","last_change":"2023-06-08T03:51:49.653715+0000","last_active":"2023-06-08T03:51:50.639581+0000","last_peered":"2023-06-08T03:51:50.639581+0000","last_clean":"2023-06-08T03:51:50.639581+0000","last_became_active":"2023-06-08T03:51:49.653418+0000","last_became_peered":"2023-06-08T03:51:49.653418+0000","last_unstale":"2023-06-08T03:51:50.639581+0000","last_undegraded":"2023-06-08T03:51:50.639581+0000","last_fullsized":"2023-06-08T03:51:50.639581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:19:14.834108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402671+0000","last_change":"2023-06-08T03:51:49.248236+0000","last_active":"2023-06-08T03:51:51.402671+0000","last_peered":"2023-06-08T03:51:51.402671+0000","last_clean":"2023-06-08T03:51:51.402671+0000","last_became_active":"2023-06-08T03:51:49.248031+0000","last_became_peered":"2023-06-08T03:51:49.248031+0000","last_unstale":"2023-06-08T03:51:51.402671+0000","last_undegraded":"2023-06-08T03:51:51.402671+0000","last_fullsized":"2023-06-08T03:51:51.402671+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:30:45.030186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639359+0000","last_change":"2023-06-08T03:51:49.662601+0000","last_active":"2023-06-08T03:51:50.639359+0000","last_peered":"2023-06-08T03:51:50.639359+0000","last_clean":"2023-06-08T03:51:50.639359+0000","last_became_active":"2023-06-08T03:51:49.662382+0000","last_became_peered":"2023-06-08T03:51:49.662382+0000","last_unstale":"2023-06-08T03:51:50.639359+0000","last_undegraded":"2023-06-08T03:51:50.639359+0000","last_fullsized":"2023-06-08T03:51:50.639359+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:55:51.849849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639694+0000","last_change":"2023-06-08T03:51:49.248130+0000","last_active":"2023-06-08T03:51:50.639694+0000","last_peered":"2023-06-08T03:51:50.639694+0000","last_clean":"2023-06-08T03:51:50.639694+0000","last_became_active":"2023-06-08T03:51:49.247852+0000","last_became_peered":"2023-06-08T03:51:49.247852+0000","last_unstale":"2023-06-08T03:51:50.639694+0000","last_undegraded":"2023-06-08T03:51:50.639694+0000","last_fullsized":"2023-06-08T03:51:50.639694+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:34:24.051362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402021+0000","last_change":"2023-06-08T03:51:49.651255+0000","last_active":"2023-06-08T03:51:51.402021+0000","last_peered":"2023-06-08T03:51:51.402021+0000","last_clean":"2023-06-08T03:51:51.402021+0000","last_became_active":"2023-06-08T03:51:49.651013+0000","last_became_peered":"2023-06-08T03:51:49.651013+0000","last_unstale":"2023-06-08T03:51:51.402021+0000","last_undegraded":"2023-06-08T03:51:51.402021+0000","last_fullsized":"2023-06-08T03:51:51.402021+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:18:50.110716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"37'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083765+0000","last_change":"2023-06-08T03:51:49.248251+0000","last_active":"2023-06-08T03:51:51.083765+0000","last_peered":"2023-06-08T03:51:51.083765+0000","last_clean":"2023-06-08T03:51:51.083765+0000","last_became_active":"2023-06-08T03:51:49.247969+0000","last_became_peered":"2023-06-08T03:51:49.247969+0000","last_unstale":"2023-06-08T03:51:51.083765+0000","last_undegraded":"2023-06-08T03:51:51.083765+0000","last_fullsized":"2023-06-08T03:51:51.083765+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:07:00.537056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639815+0000","last_change":"2023-06-08T03:51:49.654708+0000","last_active":"2023-06-08T03:51:50.639815+0000","last_peered":"2023-06-08T03:51:50.639815+0000","last_clean":"2023-06-08T03:51:50.639815+0000","last_became_active":"2023-06-08T03:51:49.654448+0000","last_became_peered":"2023-06-08T03:51:49.654448+0000","last_unstale":"2023-06-08T03:51:50.639815+0000","last_undegraded":"2023-06-08T03:51:50.639815+0000","last_fullsized":"2023-06-08T03:51:50.639815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:19:43.000984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402674+0000","last_change":"2023-06-08T03:51:49.246913+0000","last_active":"2023-06-08T03:51:51.402674+0000","last_peered":"2023-06-08T03:51:51.402674+0000","last_clean":"2023-06-08T03:51:51.402674+0000","last_became_active":"2023-06-08T03:51:49.246632+0000","last_became_peered":"2023-06-08T03:51:49.246632+0000","last_unstale":"2023-06-08T03:51:51.402674+0000","last_undegraded":"2023-06-08T03:51:51.402674+0000","last_fullsized":"2023-06-08T03:51:51.402674+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:57:39.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640084+0000","last_change":"2023-06-08T03:51:49.657622+0000","last_active":"2023-06-08T03:51:50.640084+0000","last_peered":"2023-06-08T03:51:50.640084+0000","last_clean":"2023-06-08T03:51:50.640084+0000","last_became_active":"2023-06-08T03:51:49.657361+0000","last_became_peered":"2023-06-08T03:51:49.657361+0000","last_unstale":"2023-06-08T03:51:50.640084+0000","last_undegraded":"2023-06-08T03:51:50.640084+0000","last_fullsized":"2023-06-08T03:51:50.640084+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:58:36.955240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402987+0000","last_change":"2023-06-08T03:51:49.247759+0000","last_active":"2023-06-08T03:51:51.402987+0000","last_peered":"2023-06-08T03:51:51.402987+0000","last_clean":"2023-06-08T03:51:51.402987+0000","last_became_active":"2023-06-08T03:51:49.247490+0000","last_became_peered":"2023-06-08T03:51:49.247490+0000","last_unstale":"2023-06-08T03:51:51.402987+0000","last_undegraded":"2023-06-08T03:51:51.402987+0000","last_fullsized":"2023-06-08T03:51:51.402987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:27:17.290056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402198+0000","last_change":"2023-06-08T03:51:49.661456+0000","last_active":"2023-06-08T03:51:51.402198+0000","last_peered":"2023-06-08T03:51:51.402198+0000","last_clean":"2023-06-08T03:51:51.402198+0000","last_became_active":"2023-06-08T03:51:49.660862+0000","last_became_peered":"2023-06-08T03:51:49.660862+0000","last_unstale":"2023-06-08T03:51:51.402198+0000","last_undegraded":"2023-06-08T03:51:51.402198+0000","last_fullsized":"2023-06-08T03:51:51.402198+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:45:07.842972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"37'1","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402781+0000","last_change":"2023-06-08T03:51:49.246164+0000","last_active":"2023-06-08T03:51:51.402781+0000","last_peered":"2023-06-08T03:51:51.402781+0000","last_clean":"2023-06-08T03:51:51.402781+0000","last_became_active":"2023-06-08T03:51:49.245893+0000","last_became_peered":"2023-06-08T03:51:49.245893+0000","last_unstale":"2023-06-08T03:51:51.402781+0000","last_undegraded":"2023-06-08T03:51:51.402781+0000","last_fullsized":"2023-06-08T03:51:51.402781+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T03:54:13.881533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402834+0000","last_change":"2023-06-08T03:51:49.656445+0000","last_active":"2023-06-08T03:51:51.402834+0000","last_peered":"2023-06-08T03:51:51.402834+0000","last_clean":"2023-06-08T03:51:51.402834+0000","last_became_active":"2023-06-08T03:51:49.656209+0000","last_became_peered":"2023-06-08T03:51:49.656209+0000","last_unstale":"2023-06-08T03:51:51.402834+0000","last_undegraded":"2023-06-08T03:51:51.402834+0000","last_fullsized":"2023-06-08T03:51:51.402834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:29:28.036266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639916+0000","last_change":"2023-06-08T03:51:49.247889+0000","last_active":"2023-06-08T03:51:50.639916+0000","last_peered":"2023-06-08T03:51:50.639916+0000","last_clean":"2023-06-08T03:51:50.639916+0000","last_became_active":"2023-06-08T03:51:49.247578+0000","last_became_peered":"2023-06-08T03:51:49.247578+0000","last_unstale":"2023-06-08T03:51:50.639916+0000","last_undegraded":"2023-06-08T03:51:50.639916+0000","last_fullsized":"2023-06-08T03:51:50.639916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:08.496488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083476+0000","last_change":"2023-06-08T03:51:49.658373+0000","last_active":"2023-06-08T03:51:51.083476+0000","last_peered":"2023-06-08T03:51:51.083476+0000","last_clean":"2023-06-08T03:51:51.083476+0000","last_became_active":"2023-06-08T03:51:49.658029+0000","last_became_peered":"2023-06-08T03:51:49.658029+0000","last_unstale":"2023-06-08T03:51:51.083476+0000","last_undegraded":"2023-06-08T03:51:51.083476+0000","last_fullsized":"2023-06-08T03:51:51.083476+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:41:37.880512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083564+0000","last_change":"2023-06-08T03:51:49.243549+0000","last_active":"2023-06-08T03:51:51.083564+0000","last_peered":"2023-06-08T03:51:51.083564+0000","last_clean":"2023-06-08T03:51:51.083564+0000","last_became_active":"2023-06-08T03:51:49.242639+0000","last_became_peered":"2023-06-08T03:51:49.242639+0000","last_unstale":"2023-06-08T03:51:51.083564+0000","last_undegraded":"2023-06-08T03:51:51.083564+0000","last_fullsized":"2023-06-08T03:51:51.083564+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:15:00.238248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639963+0000","last_change":"2023-06-08T03:51:49.656632+0000","last_active":"2023-06-08T03:51:50.639963+0000","last_peered":"2023-06-08T03:51:50.639963+0000","last_clean":"2023-06-08T03:51:50.639963+0000","last_became_active":"2023-06-08T03:51:49.656198+0000","last_became_peered":"2023-06-08T03:51:49.656198+0000","last_unstale":"2023-06-08T03:51:50.639963+0000","last_undegraded":"2023-06-08T03:51:50.639963+0000","last_fullsized":"2023-06-08T03:51:50.639963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:23:18.727290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402495+0000","last_change":"2023-06-08T03:51:49.248194+0000","last_active":"2023-06-08T03:51:51.402495+0000","last_peered":"2023-06-08T03:51:51.402495+0000","last_clean":"2023-06-08T03:51:51.402495+0000","last_became_active":"2023-06-08T03:51:49.247677+0000","last_became_peered":"2023-06-08T03:51:49.247677+0000","last_unstale":"2023-06-08T03:51:51.402495+0000","last_undegraded":"2023-06-08T03:51:51.402495+0000","last_fullsized":"2023-06-08T03:51:51.402495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:27:39.054568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402176+0000","last_change":"2023-06-08T03:51:49.648637+0000","last_active":"2023-06-08T03:51:51.402176+0000","last_peered":"2023-06-08T03:51:51.402176+0000","last_clean":"2023-06-08T03:51:51.402176+0000","last_became_active":"2023-06-08T03:51:49.648378+0000","last_became_peered":"2023-06-08T03:51:49.648378+0000","last_unstale":"2023-06-08T03:51:51.402176+0000","last_undegraded":"2023-06-08T03:51:51.402176+0000","last_fullsized":"2023-06-08T03:51:51.402176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:00:11.329673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083284+0000","last_change":"2023-06-08T03:51:49.247327+0000","last_active":"2023-06-08T03:51:51.083284+0000","last_peered":"2023-06-08T03:51:51.083284+0000","last_clean":"2023-06-08T03:51:51.083284+0000","last_became_active":"2023-06-08T03:51:49.247039+0000","last_became_peered":"2023-06-08T03:51:49.247039+0000","last_unstale":"2023-06-08T03:51:51.083284+0000","last_undegraded":"2023-06-08T03:51:51.083284+0000","last_fullsized":"2023-06-08T03:51:51.083284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:56:11.190578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402116+0000","last_change":"2023-06-08T03:51:49.656775+0000","last_active":"2023-06-08T03:51:51.402116+0000","last_peered":"2023-06-08T03:51:51.402116+0000","last_clean":"2023-06-08T03:51:51.402116+0000","last_became_active":"2023-06-08T03:51:49.656454+0000","last_became_peered":"2023-06-08T03:51:49.656454+0000","last_unstale":"2023-06-08T03:51:51.402116+0000","last_undegraded":"2023-06-08T03:51:51.402116+0000","last_fullsized":"2023-06-08T03:51:51.402116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T15:17:10.588386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638480+0000","last_change":"2023-06-08T03:51:49.248743+0000","last_active":"2023-06-08T03:51:50.638480+0000","last_peered":"2023-06-08T03:51:50.638480+0000","last_clean":"2023-06-08T03:51:50.638480+0000","last_became_active":"2023-06-08T03:51:49.247400+0000","last_became_peered":"2023-06-08T03:51:49.247400+0000","last_unstale":"2023-06-08T03:51:50.638480+0000","last_undegraded":"2023-06-08T03:51:50.638480+0000","last_fullsized":"2023-06-08T03:51:50.638480+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:07:33.995899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402300+0000","last_change":"2023-06-08T03:51:49.653916+0000","last_active":"2023-06-08T03:51:51.402300+0000","last_peered":"2023-06-08T03:51:51.402300+0000","last_clean":"2023-06-08T03:51:51.402300+0000","last_became_active":"2023-06-08T03:51:49.653598+0000","last_became_peered":"2023-06-08T03:51:49.653598+0000","last_unstale":"2023-06-08T03:51:51.402300+0000","last_undegraded":"2023-06-08T03:51:51.402300+0000","last_fullsized":"2023-06-08T03:51:51.402300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:44:05.533860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639503+0000","last_change":"2023-06-08T03:51:49.254412+0000","last_active":"2023-06-08T03:51:50.639503+0000","last_peered":"2023-06-08T03:51:50.639503+0000","last_clean":"2023-06-08T03:51:50.639503+0000","last_became_active":"2023-06-08T03:51:49.254165+0000","last_became_peered":"2023-06-08T03:51:49.254165+0000","last_unstale":"2023-06-08T03:51:50.639503+0000","last_undegraded":"2023-06-08T03:51:50.639503+0000","last_fullsized":"2023-06-08T03:51:50.639503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:56:24.030853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402044+0000","last_change":"2023-06-08T03:50:19.804887+0000","last_active":"2023-06-08T03:51:51.402044+0000","last_peered":"2023-06-08T03:51:51.402044+0000","last_clean":"2023-06-08T03:51:51.402044+0000","last_became_active":"2023-06-08T03:50:19.804546+0000","last_became_peered":"2023-06-08T03:50:19.804546+0000","last_unstale":"2023-06-08T03:51:51.402044+0000","last_undegraded":"2023-06-08T03:51:51.402044+0000","last_fullsized":"2023-06-08T03:51:51.402044+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:22:59.848957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083461+0000","last_change":"2023-06-08T03:51:49.658733+0000","last_active":"2023-06-08T03:51:51.083461+0000","last_peered":"2023-06-08T03:51:51.083461+0000","last_clean":"2023-06-08T03:51:51.083461+0000","last_became_active":"2023-06-08T03:51:49.658431+0000","last_became_peered":"2023-06-08T03:51:49.658431+0000","last_unstale":"2023-06-08T03:51:51.083461+0000","last_undegraded":"2023-06-08T03:51:51.083461+0000","last_fullsized":"2023-06-08T03:51:51.083461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:29:48.976070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402874+0000","last_change":"2023-06-08T03:51:49.250518+0000","last_active":"2023-06-08T03:51:51.402874+0000","last_peered":"2023-06-08T03:51:51.402874+0000","last_clean":"2023-06-08T03:51:51.402874+0000","last_became_active":"2023-06-08T03:51:49.250251+0000","last_became_peered":"2023-06-08T03:51:49.250251+0000","last_unstale":"2023-06-08T03:51:51.402874+0000","last_undegraded":"2023-06-08T03:51:51.402874+0000","last_fullsized":"2023-06-08T03:51:51.402874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:04:21.297093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640105+0000","last_change":"2023-06-08T03:50:19.172271+0000","last_active":"2023-06-08T03:51:50.640105+0000","last_peered":"2023-06-08T03:51:50.640105+0000","last_clean":"2023-06-08T03:51:50.640105+0000","last_became_active":"2023-06-08T03:50:19.171917+0000","last_became_peered":"2023-06-08T03:50:19.171917+0000","last_unstale":"2023-06-08T03:51:50.640105+0000","last_undegraded":"2023-06-08T03:51:50.640105+0000","last_fullsized":"2023-06-08T03:51:50.640105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:16:27.815046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639045+0000","last_change":"2023-06-08T03:51:49.662049+0000","last_active":"2023-06-08T03:51:50.639045+0000","last_peered":"2023-06-08T03:51:50.639045+0000","last_clean":"2023-06-08T03:51:50.639045+0000","last_became_active":"2023-06-08T03:51:49.661549+0000","last_became_peered":"2023-06-08T03:51:49.661549+0000","last_unstale":"2023-06-08T03:51:50.639045+0000","last_undegraded":"2023-06-08T03:51:50.639045+0000","last_fullsized":"2023-06-08T03:51:50.639045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:20:12.830253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.640007+0000","last_change":"2023-06-08T03:51:49.246833+0000","last_active":"2023-06-08T03:51:50.640007+0000","last_peered":"2023-06-08T03:51:50.640007+0000","last_clean":"2023-06-08T03:51:50.640007+0000","last_became_active":"2023-06-08T03:51:49.246565+0000","last_became_peered":"2023-06-08T03:51:49.246565+0000","last_unstale":"2023-06-08T03:51:50.640007+0000","last_undegraded":"2023-06-08T03:51:50.640007+0000","last_fullsized":"2023-06-08T03:51:50.640007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:50:04.802330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083691+0000","last_change":"2023-06-08T03:50:19.171073+0000","last_active":"2023-06-08T03:51:51.083691+0000","last_peered":"2023-06-08T03:51:51.083691+0000","last_clean":"2023-06-08T03:51:51.083691+0000","last_became_active":"2023-06-08T03:50:19.170775+0000","last_became_peered":"2023-06-08T03:50:19.170775+0000","last_unstale":"2023-06-08T03:51:51.083691+0000","last_undegraded":"2023-06-08T03:51:51.083691+0000","last_fullsized":"2023-06-08T03:51:51.083691+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:48:36.714078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":195,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638526+0000","last_change":"2023-06-08T03:50:19.167800+0000","last_active":"2023-06-08T03:51:50.638526+0000","last_peered":"2023-06-08T03:51:50.638526+0000","last_clean":"2023-06-08T03:51:50.638526+0000","last_became_active":"2023-06-08T03:50:19.167412+0000","last_became_peered":"2023-06-08T03:50:19.167412+0000","last_unstale":"2023-06-08T03:51:50.638526+0000","last_undegraded":"2023-06-08T03:51:50.638526+0000","last_fullsized":"2023-06-08T03:51:50.638526+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:29:15.610496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.083770+0000","last_change":"2023-06-08T03:51:49.664632+0000","last_active":"2023-06-08T03:51:51.083770+0000","last_peered":"2023-06-08T03:51:51.083770+0000","last_clean":"2023-06-08T03:51:51.083770+0000","last_became_active":"2023-06-08T03:51:49.664384+0000","last_became_peered":"2023-06-08T03:51:49.664384+0000","last_unstale":"2023-06-08T03:51:51.083770+0000","last_undegraded":"2023-06-08T03:51:51.083770+0000","last_fullsized":"2023-06-08T03:51:51.083770+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:27:22.320056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402098+0000","last_change":"2023-06-08T03:51:49.247945+0000","last_active":"2023-06-08T03:51:51.402098+0000","last_peered":"2023-06-08T03:51:51.402098+0000","last_clean":"2023-06-08T03:51:51.402098+0000","last_became_active":"2023-06-08T03:51:49.247630+0000","last_became_peered":"2023-06-08T03:51:49.247630+0000","last_unstale":"2023-06-08T03:51:51.402098+0000","last_undegraded":"2023-06-08T03:51:51.402098+0000","last_fullsized":"2023-06-08T03:51:51.402098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T04:51:08.496488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402580+0000","last_change":"2023-06-08T03:50:19.171834+0000","last_active":"2023-06-08T03:51:51.402580+0000","last_peered":"2023-06-08T03:51:51.402580+0000","last_clean":"2023-06-08T03:51:51.402580+0000","last_became_active":"2023-06-08T03:50:19.171541+0000","last_became_peered":"2023-06-08T03:50:19.171541+0000","last_unstale":"2023-06-08T03:51:51.402580+0000","last_undegraded":"2023-06-08T03:51:51.402580+0000","last_fullsized":"2023-06-08T03:51:51.402580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T06:16:35.884773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402672+0000","last_change":"2023-06-08T03:51:49.647218+0000","last_active":"2023-06-08T03:51:51.402672+0000","last_peered":"2023-06-08T03:51:51.402672+0000","last_clean":"2023-06-08T03:51:51.402672+0000","last_became_active":"2023-06-08T03:51:49.646954+0000","last_became_peered":"2023-06-08T03:51:49.646954+0000","last_unstale":"2023-06-08T03:51:51.402672+0000","last_undegraded":"2023-06-08T03:51:51.402672+0000","last_fullsized":"2023-06-08T03:51:51.402672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T10:15:49.612923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"37'2","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639064+0000","last_change":"2023-06-08T03:51:49.254182+0000","last_active":"2023-06-08T03:51:50.639064+0000","last_peered":"2023-06-08T03:51:50.639064+0000","last_clean":"2023-06-08T03:51:50.639064+0000","last_became_active":"2023-06-08T03:51:49.253982+0000","last_became_peered":"2023-06-08T03:51:49.253982+0000","last_unstale":"2023-06-08T03:51:50.639064+0000","last_undegraded":"2023-06-08T03:51:50.639064+0000","last_fullsized":"2023-06-08T03:51:50.639064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:09:24.316204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402556+0000","last_change":"2023-06-08T03:50:19.170601+0000","last_active":"2023-06-08T03:51:51.402556+0000","last_peered":"2023-06-08T03:51:51.402556+0000","last_clean":"2023-06-08T03:51:51.402556+0000","last_became_active":"2023-06-08T03:50:19.170344+0000","last_became_peered":"2023-06-08T03:50:19.170344+0000","last_unstale":"2023-06-08T03:51:51.402556+0000","last_undegraded":"2023-06-08T03:51:51.402556+0000","last_fullsized":"2023-06-08T03:51:51.402556+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:30:37.760942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638992+0000","last_change":"2023-06-08T03:51:49.658506+0000","last_active":"2023-06-08T03:51:50.638992+0000","last_peered":"2023-06-08T03:51:50.638992+0000","last_clean":"2023-06-08T03:51:50.638992+0000","last_became_active":"2023-06-08T03:51:49.658233+0000","last_became_peered":"2023-06-08T03:51:49.658233+0000","last_unstale":"2023-06-08T03:51:50.638992+0000","last_undegraded":"2023-06-08T03:51:50.638992+0000","last_fullsized":"2023-06-08T03:51:50.638992+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T07:21:06.247391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402768+0000","last_change":"2023-06-08T03:51:49.254869+0000","last_active":"2023-06-08T03:51:51.402768+0000","last_peered":"2023-06-08T03:51:51.402768+0000","last_clean":"2023-06-08T03:51:51.402768+0000","last_became_active":"2023-06-08T03:51:49.254607+0000","last_became_peered":"2023-06-08T03:51:49.254607+0000","last_unstale":"2023-06-08T03:51:51.402768+0000","last_undegraded":"2023-06-08T03:51:51.402768+0000","last_fullsized":"2023-06-08T03:51:51.402768+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:37:28.919045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.638651+0000","last_change":"2023-06-08T03:50:19.805195+0000","last_active":"2023-06-08T03:51:50.638651+0000","last_peered":"2023-06-08T03:51:50.638651+0000","last_clean":"2023-06-08T03:51:50.638651+0000","last_became_active":"2023-06-08T03:50:19.804708+0000","last_became_peered":"2023-06-08T03:50:19.804708+0000","last_unstale":"2023-06-08T03:51:50.638651+0000","last_undegraded":"2023-06-08T03:51:50.638651+0000","last_fullsized":"2023-06-08T03:51:50.638651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T05:21:16.612392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402527+0000","last_change":"2023-06-08T03:51:49.655690+0000","last_active":"2023-06-08T03:51:51.402527+0000","last_peered":"2023-06-08T03:51:51.402527+0000","last_clean":"2023-06-08T03:51:51.402527+0000","last_became_active":"2023-06-08T03:51:49.655350+0000","last_became_peered":"2023-06-08T03:51:49.655350+0000","last_unstale":"2023-06-08T03:51:51.402527+0000","last_undegraded":"2023-06-08T03:51:51.402527+0000","last_fullsized":"2023-06-08T03:51:51.402527+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:58:06.785641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639698+0000","last_change":"2023-06-08T03:51:49.252638+0000","last_active":"2023-06-08T03:51:50.639698+0000","last_peered":"2023-06-08T03:51:50.639698+0000","last_clean":"2023-06-08T03:51:50.639698+0000","last_became_active":"2023-06-08T03:51:49.252337+0000","last_became_peered":"2023-06-08T03:51:49.252337+0000","last_unstale":"2023-06-08T03:51:50.639698+0000","last_undegraded":"2023-06-08T03:51:50.639698+0000","last_fullsized":"2023-06-08T03:51:50.639698+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:48:47.328921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639353+0000","last_change":"2023-06-08T03:50:19.805012+0000","last_active":"2023-06-08T03:51:50.639353+0000","last_peered":"2023-06-08T03:51:50.639353+0000","last_clean":"2023-06-08T03:51:50.639353+0000","last_became_active":"2023-06-08T03:50:19.804722+0000","last_became_peered":"2023-06-08T03:50:19.804722+0000","last_unstale":"2023-06-08T03:51:50.639353+0000","last_undegraded":"2023-06-08T03:51:50.639353+0000","last_fullsized":"2023-06-08T03:51:50.639353+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T11:31:40.068465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402157+0000","last_change":"2023-06-08T03:51:49.656784+0000","last_active":"2023-06-08T03:51:51.402157+0000","last_peered":"2023-06-08T03:51:51.402157+0000","last_clean":"2023-06-08T03:51:51.402157+0000","last_became_active":"2023-06-08T03:51:49.656493+0000","last_became_peered":"2023-06-08T03:51:49.656493+0000","last_unstale":"2023-06-08T03:51:51.402157+0000","last_undegraded":"2023-06-08T03:51:51.402157+0000","last_fullsized":"2023-06-08T03:51:51.402157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:49.225956+0000","last_clean_scrub_stamp":"2023-06-08T03:51:49.225956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T12:23:56.939004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:50.639494+0000","last_change":"2023-06-08T03:51:49.242827+0000","last_active":"2023-06-08T03:51:50.639494+0000","last_peered":"2023-06-08T03:51:50.639494+0000","last_clean":"2023-06-08T03:51:50.639494+0000","last_became_active":"2023-06-08T03:51:49.242285+0000","last_became_peered":"2023-06-08T03:51:49.242285+0000","last_unstale":"2023-06-08T03:51:50.639494+0000","last_undegraded":"2023-06-08T03:51:50.639494+0000","last_fullsized":"2023-06-08T03:51:50.639494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:51:48.221873+0000","last_clean_scrub_stamp":"2023-06-08T03:51:48.221873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T14:13:06.254615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":51,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-08T03:51:51.402745+0000","last_change":"2023-06-08T03:50:19.805107+0000","last_active":"2023-06-08T03:51:51.402745+0000","last_peered":"2023-06-08T03:51:51.402745+0000","last_clean":"2023-06-08T03:51:51.402745+0000","last_became_active":"2023-06-08T03:50:19.804754+0000","last_became_peered":"2023-06-08T03:50:19.804754+0000","last_unstale":"2023-06-08T03:51:51.402745+0000","last_undegraded":"2023-06-08T03:51:51.402745+0000","last_fullsized":"2023-06-08T03:51:51.402745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-08T03:50:18.152517+0000","last_clean_scrub_stamp":"2023-06-08T03:50:18.152517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-09T13:42:16.265675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":193786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.438},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":11.465999999999999,"5min":11.465999999999999,"15min":11.465999999999999},"last":0.72399999999999998}]},{"osd":1,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.9119999999999999,"5min":3.9119999999999999,"15min":3.9119999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":7.444,"5min":7.444,"15min":7.444},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.628}]},{"osd":4,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.503}]},{"osd":6,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.47299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":446464,"data_stored":194368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":6.2569999999999997,"5min":6.2569999999999997,"15min":6.2569999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.9989999999999997,"5min":5.9989999999999997,"15min":5.9989999999999997},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.1230000000000002,"5min":6.1230000000000002,"15min":6.1230000000000002},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.497},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.57399999999999995}]},{"osd":5,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jun 8 03:51:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.58599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":193822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.17999999999999999,"5min":0.17999999999999999,"15min":0.17999999999999999},"max":{"1min":21.992000000000001,"5min":21.992000000000001,"15min":21.992000000000001},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":5.2400000000000002,"5min":5.2400000000000002,"15min":5.2400000000000002},"last":0.248}]},{"osd":2,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":5.4109999999999996,"5min":5.4109999999999996,"15min":5.4109999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.2690000000000001,"5min":5.2690000000000001,"15min":5.2690000000000001},"last":0.33900000000000002}]},{"osd":3,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":15.984999999999999,"5min":15.984999999999999,"15min":15.984999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":18.992000000000001,"5min":18.992000000000001,"15min":18.992000000000001},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":10.987,"5min":10.987,"15min":10.987},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":15.955,"5min":15.955,"15min":15.955},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.325,"15min":1.325},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":10.962999999999999,"5min":10.962999999999999,"15min":10.962999999999999},"last":0.443},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":11.023,"5min":11.023,"15min":11.023},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":10.928000000000001,"5min":10.928000000000001,"15min":10.928000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":11.051,"5min":11.051,"15min":11.051},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.923,"5min":15.923,"15min":15.923},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.696,"5min":1.696,"15min":1.696},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":18.943999999999999,"5min":18.943999999999999,"15min":18.943999999999999},"last":0.55300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":446464,"data_stored":194424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.31},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.26000000000000001}]},{"osd":2,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.374},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.627}]},{"osd":5,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":186869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.376},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":0.29699999999999999}]},{"osd":1,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.313},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":5.3929999999999998,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.5149999999999997,"5min":6.5149999999999997,"15min":6.5149999999999997},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.34399999999999997}]},{"osd":4,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.3630000000000004,"5min":5.3630000000000004,"15min":5.3630000000000004},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":6.4210000000000003,"5min":6.4210000000000003,"15min":6.4210000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.42499999999999999}]},{"osd":6,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.3360000000000003,"5min":5.3360000000000003,"15min":5.3360000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.4589999999999996,"5min":6.4589999999999996,"15min":6.4589999999999996},"last":0.60599999999999998}]},{"osd":7,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.628}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":1044480,"data_stored":785684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"last":0.20899999999999999}]},{"osd":1,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.32600000000000001}]},{"osd":5,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48699999999999999}]},{"osd":6,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.503}]},{"osd":7,"last update":"Thu Jun 8 03:51:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.38}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":458752,"data_stored":195408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.629},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.47499999999999998}]},{"osd":7,"last update":"Thu Jun 8 03:51:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.61099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":784142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.377},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.502},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jun 8 03:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-08T03:51:54.105 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-08T03:51:54.105 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-06-08T03:51:54.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.261+0000 7f9cc86ee700 1 Processor -- start 2023-06-08T03:51:54.265 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.261+0000 7f9cc86ee700 1 -- start start 2023-06-08T03:51:54.266 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc86ee700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9cc00ca360 0x7f9cc00cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc86ee700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc00cc900 con 0x7f9cc0158d50 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc86ee700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc00cd7b0 con 0x7f9cc00cc4c0 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc86ee700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc00cda50 con 0x7f9cc00ca360 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc5c89700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9cc00ca360 0x7f9cc00cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc648a700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9cc0158d50 0x7f9cc015d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32914/0 (socket says 172.21.15.79:32914) 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc648a700 1 -- 172.21.15.79:0/1635307894 learned_addr learned my addr 172.21.15.79:0/1635307894 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:54.267 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc5c89700 1 -- 172.21.15.79:0/1635307894 >> v1:172.21.15.156:6789/0 conn(0x7f9cc00cc4c0 legacy=0x7f9cc0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:54.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc5c89700 1 -- 172.21.15.79:0/1635307894 >> v1:172.21.15.79:6789/0 conn(0x7f9cc0158d50 legacy=0x7f9cc015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc5c89700 1 -- 172.21.15.79:0/1635307894 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc00cdd30 con 0x7f9cc00ca360 2023-06-08T03:51:54.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.262+0000 7f9cc5488700 1 -- 172.21.15.79:0/1635307894 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546415399 0 0) 0x7f9cc00cc900 con 0x7f9cc0158d50 2023-06-08T03:51:54.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc5c89700 1 --2- 172.21.15.79:0/1635307894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc00cbf80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009840 tx=0x7f9cbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb392fb5a2292b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:54.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc5488700 1 -- 172.21.15.79:0/1635307894 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004230 con 0x7f9cc00ca360 2023-06-08T03:51:54.268 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc5488700 1 -- 172.21.15.79:0/1635307894 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc004390 con 0x7f9cc00ca360 2023-06-08T03:51:54.269 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc5488700 1 -- 172.21.15.79:0/1635307894 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0036a0 con 0x7f9cc00ca360 2023-06-08T03:51:54.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1635307894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 msgr2=0x7f9cc00cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.270 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1635307894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc00cbf80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009840 tx=0x7f9cbc009b50 comp rx=0 tx=0).stop 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1635307894 shutdown_connections 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1635307894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc00cbf80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1635307894 >> 172.21.15.79:0/1635307894 conn(0x7f9cc00b1d30 msgr2=0x7f9cc006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1635307894 shutdown_connections 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.263+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1635307894 wait complete. 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.264+0000 7f9cc86ee700 1 Processor -- start 2023-06-08T03:51:54.271 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 -- start start 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc01f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cc4c0 0x7f9cc01f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc0158d50 0x7f9cc00cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc014cdc0 con 0x7f9cc00ca360 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc01f5f20 con 0x7f9cc00cc4c0 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc86ee700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc00b0d90 con 0x7f9cc0158d50 2023-06-08T03:51:54.272 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc648a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc01f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc648a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc01f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37676/0 (socket says 172.21.15.79:37676) 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.265+0000 7f9cc648a700 1 -- 172.21.15.79:0/1289175914 learned_addr learned my addr 172.21.15.79:0/1289175914 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc648a700 1 -- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc0158d50 msgr2=0x7f9cc00cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc648a700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc0158d50 0x7f9cc00cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc648a700 1 -- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cc4c0 msgr2=0x7f9cc01f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc648a700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cc4c0 0x7f9cc01f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.273 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc648a700 1 -- 172.21.15.79:0/1289175914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc0094a0 con 0x7f9cc00ca360 2023-06-08T03:51:54.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc648a700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc01f59e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00011e0 tx=0x7f9cb000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:54.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0004c80 con 0x7f9cc00ca360 2023-06-08T03:51:54.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb0004e00 con 0x7f9cc00ca360 2023-06-08T03:51:54.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc00af8a0 con 0x7f9cc00ca360 2023-06-08T03:51:54.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0007310 con 0x7f9cc00ca360 2023-06-08T03:51:54.274 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.266+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc00afe60 con 0x7f9cc00ca360 2023-06-08T03:51:54.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.267+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc00ab8c0 con 0x7f9cc00ca360 2023-06-08T03:51:54.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.267+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9cb0007470 con 0x7f9cc00ca360 2023-06-08T03:51:54.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.268+0000 7f9cb6ffd700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9ca406ace0 0x7f9ca406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.275 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.268+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9cb0089470 con 0x7f9cc00ca360 2023-06-08T03:51:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.268+0000 7f9cc5c89700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9ca406ace0 0x7f9ca406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.269+0000 7f9cc5c89700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9ca406ace0 0x7f9ca406d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00b0530 tx=0x7f9cbc013040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.272+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9cb0054350 con 0x7f9cc00ca360 2023-06-08T03:51:54.540 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.536+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9cc0073e90 con 0x7f9cc00ca360 2023-06-08T03:51:54.543 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.540+0000 7f9cb6ffd700 1 -- 172.21.15.79:0/1289175914 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9cb0057970 con 0x7f9cc00ca360 2023-06-08T03:51:54.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9ca406ace0 msgr2=0x7f9ca406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9ca406ace0 0x7f9ca406d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00b0530 tx=0x7f9cbc013040 comp rx=0 tx=0).stop 2023-06-08T03:51:54.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 msgr2=0x7f9cc01f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.546 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc01f59e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00011e0 tx=0x7f9cb000cae0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 shutdown_connections 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f9ca406ace0 0x7f9ca406d190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc0158d50 0x7f9cc00cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9cc00ca360 0x7f9cc01f59e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 --2- 172.21.15.79:0/1289175914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cc4c0 0x7f9cc01f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 >> 172.21.15.79:0/1289175914 conn(0x7f9cc00b1d30 msgr2=0x7f9cc00ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 shutdown_connections 2023-06-08T03:51:54.547 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.542+0000 7f9cc86ee700 1 -- 172.21.15.79:0/1289175914 wait complete. 2023-06-08T03:51:54.548 INFO:teuthology.orchestra.run.smithi079.stderr:updated caps for client.0 2023-06-08T03:51:54.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-08T03:51:54.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.677+0000 7fbce749c700 1 Processor -- start 2023-06-08T03:51:54.681 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.677+0000 7fbce749c700 1 -- start start 2023-06-08T03:51:54.682 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce749c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce015d7a0 0x7fbce00cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce749c700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbce00cd7a0 0x7fbce00b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce749c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce00ce250 con 0x7fbce00ccb30 2023-06-08T03:51:54.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce749c700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00afeb0 con 0x7fbce00cd7a0 2023-06-08T03:51:54.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce749c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00ce4a0 con 0x7fbce015d7a0 2023-06-08T03:51:54.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbce00cd7a0 0x7fbce00b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.683 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbce00cd7a0 0x7fbce00b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37688/0 (socket says 172.21.15.79:37688) 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 -- 172.21.15.79:0/253027824 learned_addr learned my addr 172.21.15.79:0/253027824 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5238700 1 --2- 172.21.15.79:0/253027824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce015d7a0 0x7fbce00cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 -- 172.21.15.79:0/253027824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce015d7a0 msgr2=0x7fbce00cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 --2- 172.21.15.79:0/253027824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce015d7a0 0x7fbce00cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 -- 172.21.15.79:0/253027824 >> v1:172.21.15.156:6789/0 conn(0x7fbce00ccb30 legacy=0x7fbce00cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.678+0000 7fbce5a39700 1 -- 172.21.15.79:0/253027824 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00ce780 con 0x7fbce00cd7a0 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce5a39700 1 --2- 172.21.15.79:0/253027824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0340 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009e50 tx=0x7fbcd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef58c3ba1bd48969 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:54.684 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbcd7fff700 1 -- 172.21.15.79:0/253027824 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd800e070 con 0x7fbce00cd7a0 2023-06-08T03:51:54.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbcd7fff700 1 -- 172.21.15.79:0/253027824 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8003a20 con 0x7fbce00cd7a0 2023-06-08T03:51:54.685 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbcd7fff700 1 -- 172.21.15.79:0/253027824 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8003dd0 con 0x7fbce00cd7a0 2023-06-08T03:51:54.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 -- 172.21.15.79:0/253027824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 msgr2=0x7fbce00b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 --2- 172.21.15.79:0/253027824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0340 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009e50 tx=0x7fbcd800b330 comp rx=0 tx=0).stop 2023-06-08T03:51:54.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 -- 172.21.15.79:0/253027824 shutdown_connections 2023-06-08T03:51:54.687 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 --2- 172.21.15.79:0/253027824 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0340 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 --2- 172.21.15.79:0/253027824 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce015d7a0 0x7fbce00cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 -- 172.21.15.79:0/253027824 >> 172.21.15.79:0/253027824 conn(0x7fbce0099960 msgr2=0x7fbce00d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 -- 172.21.15.79:0/253027824 shutdown_connections 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.679+0000 7fbce749c700 1 -- 172.21.15.79:0/253027824 wait complete. 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.680+0000 7fbce749c700 1 Processor -- start 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 -- start start 2023-06-08T03:51:54.688 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce00ccb30 0x7fbce015a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce015d7a0 0x7fbce00b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce01f1e50 con 0x7fbce00cd7a0 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce006c2e0 con 0x7fbce015d7a0 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce749c700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce006ba40 con 0x7fbce00ccb30 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce4a37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.689 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce4a37700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37692/0 (socket says 172.21.15.79:37692) 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.681+0000 7fbce4a37700 1 -- 172.21.15.79:0/3402235045 learned_addr learned my addr 172.21.15.79:0/3402235045 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce4a37700 1 -- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce00ccb30 msgr2=0x7fbce015a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce5238700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce00ccb30 0x7fbce015a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce5a39700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce015d7a0 0x7fbce00b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce4a37700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce00ccb30 0x7fbce015a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce4a37700 1 -- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce015d7a0 msgr2=0x7fbce00b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.690 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce4a37700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce015d7a0 0x7fbce00b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce4a37700 1 -- 172.21.15.79:0/3402235045 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8009b30 con 0x7fbce00cd7a0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce4a37700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0e80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b470 tx=0x7fbcdc00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc011830 con 0x7fbce00cd7a0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc011990 con 0x7fbce00cd7a0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce006bd20 con 0x7fbce00cd7a0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc011d40 con 0x7fbce00cd7a0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.682+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce015a580 con 0x7fbce00cd7a0 2023-06-08T03:51:54.691 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.683+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce00cdfb0 con 0x7fbce00cd7a0 2023-06-08T03:51:54.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.683+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcdc01a400 con 0x7fbce00cd7a0 2023-06-08T03:51:54.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.684+0000 7fbcd5ffb700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:54.692 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.684+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbcdc0895d0 con 0x7fbce00cd7a0 2023-06-08T03:51:54.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.684+0000 7fbce5238700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:54.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.685+0000 7fbce5238700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbcc406ace0 0x7fbcc406d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0008850 tx=0x7fbcd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:54.693 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.688+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbcdc0545d0 con 0x7fbce00cd7a0 2023-06-08T03:51:54.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.913+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbce0073e90 con 0x7fbce00cd7a0 2023-06-08T03:51:54.917 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.913+0000 7fbcd5ffb700 1 -- 172.21.15.79:0/3402235045 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4677 (secure 0 0 0) 0x7fbcdc057bf0 con 0x7fbce00cd7a0 2023-06-08T03:51:54.918 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:51:54.918 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5000,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6836/3221038422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":3221038422},{"type":"v1","addr":"172.21.15.156:6836","nonce":3221038422}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5242,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/2525455221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6835","nonce":2525455221},{"type":"v1","addr":"172.21.15.156:6837","nonce":2525455221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6834/3327951478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3327951478},{"type":"v1","addr":"172.21.15.79:6834","nonce":3327951478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-08T03:51:49.630919+0000","modified":"2023-06-08T03:51:52.642991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5586":{"gid":5586,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6835/2809764197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6833","nonce":2809764197},{"type":"v1","addr":"172.21.15.79:6835","nonce":2809764197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-08T03:51:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbcc406ace0 msgr2=0x7fbcc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.919 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbcc406ace0 0x7fbcc406d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0008850 tx=0x7fbcd0008040 comp rx=0 tx=0).stop 2023-06-08T03:51:54.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 msgr2=0x7fbce00b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:54.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0e80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b470 tx=0x7fbcdc00b830 comp rx=0 tx=0).stop 2023-06-08T03:51:54.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 shutdown_connections 2023-06-08T03:51:54.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbce015d7a0 0x7fbce00b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fbcc406ace0 0x7fbcc406d190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.920 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbce00ccb30 0x7fbce015a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 --2- 172.21.15.79:0/3402235045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbce00cd7a0 0x7fbce00b0e80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:54.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 >> 172.21.15.79:0/3402235045 conn(0x7fbce0099960 msgr2=0x7fbce00ad2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:54.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 shutdown_connections 2023-06-08T03:51:54.921 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:54.915+0000 7fbce749c700 1 -- 172.21.15.79:0/3402235045 wait complete. 2023-06-08T03:51:54.921 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 17 2023-06-08T03:51:55.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-08T03:51:49.630919+0000', 'modified': '2023-06-08T03:51:52.642991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5586': {'gid': 5586, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6835/2809764197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6833', 'nonce': 2809764197}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 2809764197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-08T03:51:55.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-08T03:51:55.334 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-06-08T03:51:55.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.452+0000 7fd5f2e99700 1 Processor -- start 2023-06-08T03:51:55.456 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.452+0000 7fd5f2e99700 1 -- start start 2023-06-08T03:51:55.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f2e99700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd5ec158780 0x7fd5ec06b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f2e99700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 0x7fd5ec070c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f2e99700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec071170 con 0x7fd5ec0ccb30 2023-06-08T03:51:55.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f2e99700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec06e110 con 0x7fd5ec158780 2023-06-08T03:51:55.459 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f2e99700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0c9a90 con 0x7fd5ec0707d0 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5ebfff700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fd5ec0ccb30 0x7fd5ec0c9380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32942/0 (socket says 172.21.15.79:32942) 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5ebfff700 1 -- 172.21.15.79:0/4196331894 learned_addr learned my addr 172.21.15.79:0/4196331894 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/4196331894 >> v2:172.21.15.156:3300/0 conn(0x7fd5ec158780 0x7fd5ec06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903345840 0 0) 0x7fd5ec071170 con 0x7fd5ec0ccb30 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5f1436700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 0x7fd5ec070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5dc0034d0 con 0x7fd5ec0ccb30 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657228914 0 0) 0x7fd5dc0034d0 con 0x7fd5ec0ccb30 2023-06-08T03:51:55.460 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 msgr2=0x7fd5ec070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 0x7fd5ec070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 >> v2:172.21.15.156:3300/0 conn(0x7fd5ec158780 msgr2=0x7fd5ec06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 --2- 172.21.15.79:0/4196331894 >> v2:172.21.15.156:3300/0 conn(0x7fd5ec158780 0x7fd5ec06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec06da90 con 0x7fd5ec0ccb30 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 922535793 0 0) 0x7fd5d8001670 con 0x7fd5ec0ccb30 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.453+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 >> v1:172.21.15.79:6789/0 conn(0x7fd5ec0ccb30 legacy=0x7fd5ec0c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5dc003af0 0x7fd5dc003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 0x7fd5dc00a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5dc00d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec06e110 con 0x7fd5dc00ae80 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0c9a90 con 0x7fd5dc003af0 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5dc00dc10 con 0x7fd5ec0707d0 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5dc00d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5dc003af0 0x7fd5dc003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 0x7fd5dc00a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 -- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 msgr2=0x7fd5dc00a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec0707d0 0x7fd5dc00a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 -- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5dc003af0 msgr2=0x7fd5dc003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5dc003af0 0x7fd5dc003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 -- 172.21.15.79:0/4196331894 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc0034d0 con 0x7fd5dc00ae80 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5f1436700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5dc00d2e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009fc0 tx=0x7fd5dc003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e35bfacbcc7d42 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.463 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400ec30 con 0x7fd5dc00ae80 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e400f040 con 0x7fd5dc00ae80 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.454+0000 7fd5eb7fe700 1 -- 172.21.15.79:0/4196331894 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4014480 con 0x7fd5dc00ae80 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f2e99700 1 -- 172.21.15.79:0/4196331894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 msgr2=0x7fd5dc00d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/4196331894 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5dc00d2e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009fc0 tx=0x7fd5dc003780 comp rx=0 tx=0).stop 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f1436700 1 -- 172.21.15.79:0/4196331894 reap_dead start 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f2e99700 1 -- 172.21.15.79:0/4196331894 shutdown_connections 2023-06-08T03:51:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f2e99700 1 -- 172.21.15.79:0/4196331894 >> 172.21.15.79:0/4196331894 conn(0x7fd5ec0ced30 msgr2=0x7fd5ec0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f2e99700 1 -- 172.21.15.79:0/4196331894 shutdown_connections 2023-06-08T03:51:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.455+0000 7fd5f2e99700 1 -- 172.21.15.79:0/4196331894 wait complete. 2023-06-08T03:51:55.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.456+0000 7fd5f2e99700 1 Processor -- start 2023-06-08T03:51:55.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.456+0000 7fd5f2e99700 1 -- start start 2023-06-08T03:51:55.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.457+0000 7fd5f2e99700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5ec09f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.468 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.457+0000 7fd5f2e99700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec06ccb0 0x7fd5ec06d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.457+0000 7fd5f2e99700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5ec1dfc70 0x7fd5ec1e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.457+0000 7fd5f2e99700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1e1a40 con 0x7fd5dc00ae80 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.457+0000 7fd5f2e99700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec06d660 con 0x7fd5ec1dfc70 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.457+0000 7fd5f2e99700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec098b50 con 0x7fd5ec06ccb0 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5ec09f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5ec09f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37710/0 (socket says 172.21.15.79:37710) 2023-06-08T03:51:55.469 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 -- 172.21.15.79:0/3727817200 learned_addr learned my addr 172.21.15.79:0/3727817200 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f1436700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5ec1dfc70 0x7fd5ec1e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec06ccb0 0x7fd5ec06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 -- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec06ccb0 msgr2=0x7fd5ec06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec06ccb0 0x7fd5ec06d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 -- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5ec1dfc70 msgr2=0x7fd5ec1e00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5ec1dfc70 0x7fd5ec1e00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 -- 172.21.15.79:0/3727817200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec1e09a0 con 0x7fd5dc00ae80 2023-06-08T03:51:55.470 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec06ccb0 0x7fd5ec06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5ec09f470 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011ad0 tx=0x7fd5e0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5e97fa700 1 -- 172.21.15.79:0/3727817200 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000fa60 con 0x7fd5dc00ae80 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5e97fa700 1 -- 172.21.15.79:0/3727817200 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e000fbc0 con 0x7fd5dc00ae80 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec1e0c80 con 0x7fd5dc00ae80 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5e97fa700 1 -- 172.21.15.79:0/3727817200 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0014440 con 0x7fd5dc00ae80 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.458+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec1e1240 con 0x7fd5dc00ae80 2023-06-08T03:51:55.471 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.460+0000 7fd5e97fa700 1 -- 172.21.15.79:0/3727817200 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5e00145a0 con 0x7fd5dc00ae80 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.460+0000 7fd5e97fa700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd5d006ace0 0x7fd5d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.460+0000 7fd5e97fa700 1 -- 172.21.15.79:0/3727817200 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5e0089de0 con 0x7fd5dc00ae80 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.460+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd5d006ace0 0x7fd5d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.461+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd5d006ace0 0x7fd5d006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec1dfc00 tx=0x7fd5d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.462+0000 7fd5ff01eb80 1 Processor -- start 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.462+0000 7fd5ff01eb80 1 -- start start 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b447e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.472 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b44c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b447c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b43823b0 con 0x5626b44a5cb0 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b4383c60 con 0x5626b44a1fb0 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b43d6570 con 0x5626b44c4e70 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b44c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b44c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37720/0 (socket says 172.21.15.79:37720) 2023-06-08T03:51:55.473 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1455769001 learned_addr learned my addr 172.21.15.79:0/1455769001 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b447e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5f1436700 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b447c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 msgr2=0x5626b447c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b447c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 msgr2=0x5626b447e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b447e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1455769001 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b41fdd90 con 0x5626b44a5cb0 2023-06-08T03:51:55.474 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b44c4930 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007230 tx=0x7fd5d8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f943c8e9bd82763 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5d67fc700 1 -- 172.21.15.79:0/1455769001 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8016520 con 0x5626b44a5cb0 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5d67fc700 1 -- 172.21.15.79:0/1455769001 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d8016680 con 0x5626b44a5cb0 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.463+0000 7fd5d67fc700 1 -- 172.21.15.79:0/1455769001 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80169f0 con 0x5626b44a5cb0 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1455769001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 msgr2=0x5626b44c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b44c4930 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8007230 tx=0x7fd5d8013000 comp rx=0 tx=0).stop 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1455769001 shutdown_connections 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b447c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.475 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b447e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1455769001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b44c4930 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1455769001 >> 172.21.15.79:0/1455769001 conn(0x5626b447ace0 msgr2=0x5626b44c34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1455769001 shutdown_connections 2023-06-08T03:51:55.476 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.464+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1455769001 wait complete. 2023-06-08T03:51:55.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.465+0000 7fd5ff01eb80 1 -- start start 2023-06-08T03:51:55.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.465+0000 7fd5ff01eb80 1 Processor -- start 2023-06-08T03:51:55.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b44bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a5cb0 0x5626b44c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.479 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 0x5626b44a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b43cfa00 con 0x5626b44c4e70 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b436fa70 con 0x5626b44a1fb0 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b436f770 con 0x5626b44a5cb0 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 10 client.0 _ll_register_callbacks cb 0x5626b436d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 0x5626b44a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 0x5626b44a6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37728/0 (socket says 172.21.15.79:37728) 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 -- 172.21.15.79:0/1385923755 learned_addr learned my addr 172.21.15.79:0/1385923755 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.480 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b44bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a5cb0 0x5626b44c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 -- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a5cb0 msgr2=0x5626b44c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a5cb0 0x5626b44c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 -- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 msgr2=0x5626b44bdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b44bdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 -- 172.21.15.79:0/1385923755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d800f040 con 0x5626b44c4e70 2023-06-08T03:51:55.481 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5f1436700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 0x5626b44a6200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4017040 tx=0x7fd5e4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58403623de457387 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/1385923755 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4022070 con 0x5626b44c4e70 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/1385923755 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5e400ec40 con 0x5626b44c4e70 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626b4470850 con 0x5626b44c4e70 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/1385923755 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e4007000 con 0x5626b44c4e70 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 10 client.5769 fetch_fsmap learned FSMap version 17 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.466+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5626b41e6c10 con 0x5626b44c4e70 2023-06-08T03:51:55.482 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/1385923755 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626b4470850 con 0x5626b44c4e70 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/1385923755 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd5e4003fa0 con 0x5626b44c4e70 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5ff01eb80 10 client.5769 fetch_fsmap finished waiting for FSMap version 17 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5ff01eb80 10 client.5769 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5ff01eb80 20 client.5769 populate_metadata read hostname 'smithi079' 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b447bbf0 0x5626b447bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5ff01eb80 4 client.5769 mds_command: new command op to 5586 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-08T03:51:55.483 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 --> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626b43c89e0 con 0x5626b447bbf0 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b447bbf0 0x5626b447bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5cb7fe700 10 client.5769 ms_handle_connect on v2:172.21.15.79:6833/2809764197 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.467+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b447bbf0 0x5626b447bfc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1175b26bc6532ac8 server_cookie=3997991b392c12d3 in_seq=0 out_seq=0 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/1385923755 <== mds.0 v2:172.21.15.79:6833/2809764197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626b43c89e0 con 0x5626b447bbf0 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5cb7fe700 10 client.5769 handle_command_reply: tid=0 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 client.5769 shutdown 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 msgr2=0x5626b44a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.484 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 0x5626b44a6200 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4017040 tx=0x7fd5e4004b40 comp rx=0 tx=0).stop 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 shutdown_connections 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b447bbf0 0x5626b447bfc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44c4e70 0x5626b44a6200 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a1fb0 0x5626b44bdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5cb7fe700 0 client.5769 ms_handle_reset on v2:172.21.15.79:6833/2809764197 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1385923755 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a5cb0 0x5626b44c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.485 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 >> 172.21.15.79:0/1385923755 conn(0x5626b447ace0 msgr2=0x5626b44c3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 shutdown_connections 2023-06-08T03:51:55.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1385923755 wait complete. 2023-06-08T03:51:55.486 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.468+0000 7fd5ff01eb80 20 client.5769 trim_cache size 0 max 16384 2023-06-08T03:51:55.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.485+0000 7fd5ff01eb80 1 Processor -- start 2023-06-08T03:51:55.489 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.485+0000 7fd5ff01eb80 1 -- start start 2023-06-08T03:51:55.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a1fb0 0x5626b447e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b4472510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44c4e70 0x5626b44c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b42538d0 con 0x5626b44a5cb0 2023-06-08T03:51:55.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b4471ac0 con 0x5626b44c4e70 2023-06-08T03:51:55.491 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b447d360 con 0x5626b44a1fb0 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ebfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b4472510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ebfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b4472510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37734/0 (socket says 172.21.15.79:37734) 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1934653760 learned_addr learned my addr 172.21.15.79:0/1934653760 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a1fb0 0x5626b447e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5f1436700 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44c4e70 0x5626b44c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a1fb0 msgr2=0x5626b447e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.492 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a1fb0 0x5626b447e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.486+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44c4e70 msgr2=0x5626b44c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44c4e70 0x5626b44c30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ebfff700 1 -- 172.21.15.79:0/1934653760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b44c4c00 con 0x5626b44a5cb0 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5f1436700 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44c4e70 0x5626b44c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b4472510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8000fb0 tx=0x7fd5d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908c849cb07cca8d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5d6ffd700 1 -- 172.21.15.79:0/1934653760 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8028070 con 0x5626b44a5cb0 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5d6ffd700 1 -- 172.21.15.79:0/1934653760 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d8029040 con 0x5626b44a5cb0 2023-06-08T03:51:55.493 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5d6ffd700 1 -- 172.21.15.79:0/1934653760 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80169f0 con 0x5626b44a5cb0 2023-06-08T03:51:55.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1934653760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 msgr2=0x5626b4472510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b4472510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8000fb0 tx=0x7fd5d8014000 comp rx=0 tx=0).stop 2023-06-08T03:51:55.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1934653760 shutdown_connections 2023-06-08T03:51:55.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44c4e70 0x5626b44c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.497 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44a1fb0 0x5626b447e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/1934653760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a5cb0 0x5626b4472510 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1934653760 >> 172.21.15.79:0/1934653760 conn(0x5626b447ace0 msgr2=0x5626b44bd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1934653760 shutdown_connections 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.487+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/1934653760 wait complete. 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.488+0000 7fd5ff01eb80 1 -- start start 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.488+0000 7fd5ff01eb80 1 Processor -- start 2023-06-08T03:51:55.498 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 0x5626b44a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a5cb0 0x5626b44a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b44a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b44bead0 con 0x5626b44a1fb0 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b44c7140 con 0x5626b44a5cb0 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626b44a5830 con 0x5626b44c4e70 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ff01eb80 10 client.0 _ll_register_callbacks cb 0x5626b436d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-08T03:51:55.499 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 0x5626b44a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 0x5626b44a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37738/0 (socket says 172.21.15.79:37738) 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 -- 172.21.15.79:0/845714547 learned_addr learned my addr 172.21.15.79:0/845714547 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a5cb0 0x5626b44a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f1436700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b44a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 -- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 msgr2=0x5626b44a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b44a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 -- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a5cb0 msgr2=0x5626b44a88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.500 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a5cb0 0x5626b44a88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 -- 172.21.15.79:0/845714547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b41fdd90 con 0x5626b44a1fb0 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a5cb0 0x5626b44a88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5f0c35700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 0x5626b44a10b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0088070 tx=0x7fd5e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4178265f692160 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.489+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/845714547 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0056c10 con 0x5626b44a1fb0 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/845714547 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5e0072dd0 con 0x5626b44a1fb0 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626b449d150 con 0x5626b44a1fb0 2023-06-08T03:51:55.501 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/845714547 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e00772b0 con 0x5626b44a1fb0 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 10 client.5775 fetch_fsmap learned FSMap version 17 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x7fd5ec1eeb90 con 0x5626b44a1fb0 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/845714547 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626b449d150 con 0x5626b44a1fb0 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/845714547 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd5e0057f00 con 0x5626b44a1fb0 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 10 client.5775 fetch_fsmap finished waiting for FSMap version 17 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 10 client.5775 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-08T03:51:55.502 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 20 client.5775 populate_metadata read hostname 'smithi079' 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b44e0c10 0x5626b452ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 4 client.5775 mds_command: new command op to 5586 tid=0[{"prefix": "status"}] 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 --> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd5d8016680 con 0x5626b44e0c10 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b44e0c10 0x5626b452ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5cb7fe700 10 client.5775 ms_handle_connect on v2:172.21.15.79:6833/2809764197 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.490+0000 7fd5ebfff700 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b44e0c10 0x5626b452ec80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd0bb2afa1fd8dc server_cookie=bb43c67ad02b1dd5 in_seq=0 out_seq=0 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5cb7fe700 1 -- 172.21.15.79:0/845714547 <== mds.0 v2:172.21.15.79:6833/2809764197 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fd5d8016680 con 0x5626b44e0c10 2023-06-08T03:51:55.503 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5cb7fe700 10 client.5775 handle_command_reply: tid=0 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 client.5775 shutdown 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 msgr2=0x5626b44a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 0x5626b44a10b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0088070 tx=0x7fd5e00047c0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 shutdown_connections 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:6833/2809764197,v1:172.21.15.79:6835/2809764197] conn(0x5626b44e0c10 0x5626b452ec80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5626b44c4e70 0x5626b44a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.504 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5cb7fe700 0 client.5775 ms_handle_reset on v2:172.21.15.79:6833/2809764197 2023-06-08T03:51:55.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5626b44a1fb0 0x5626b44a10b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 --2- 172.21.15.79:0/845714547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5626b44a5cb0 0x5626b44a88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 >> 172.21.15.79:0/845714547 conn(0x5626b447ace0 msgr2=0x5626b447df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 shutdown_connections 2023-06-08T03:51:55.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 1 -- 172.21.15.79:0/845714547 wait complete. 2023-06-08T03:51:55.505 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5ff01eb80 20 client.5775 trim_cache size 0 max 16384 2023-06-08T03:51:55.506 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:51:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "141881e0-5474-41d9-8205-1a44c31c5e0b", 2023-06-08T03:51:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2023-06-08T03:51:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 5586, 2023-06-08T03:51:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2023-06-08T03:51:55.506 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 5.8596033749999998, 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 17, 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 38, 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 38, 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 13.129012618000001 2023-06-08T03:51:55.507 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:51:55.508 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd5d006ace0 msgr2=0x7fd5d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd5d006ace0 0x7fd5d006d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec1dfc00 tx=0x7fd5d8009040 comp rx=0 tx=0).stop 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 msgr2=0x7fd5ec09f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5ec09f470 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0011ad0 tx=0x7fd5e0011e90 comp rx=0 tx=0).stop 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 shutdown_connections 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5ec1dfc70 0x7fd5ec1e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5dc00ae80 0x7fd5ec09f470 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.509 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7fd5d006ace0 0x7fd5d006d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.491+0000 7fd5f2e99700 1 --2- 172.21.15.79:0/3727817200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd5ec06ccb0 0x7fd5ec06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.492+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 >> 172.21.15.79:0/3727817200 conn(0x7fd5ec0ced30 msgr2=0x7fd5ec06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.492+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 shutdown_connections 2023-06-08T03:51:55.510 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.492+0000 7fd5f2e99700 1 -- 172.21.15.79:0/3727817200 wait complete. 2023-06-08T03:51:55.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-08T03:51:55.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.626+0000 7f1e9bede700 1 Processor -- start 2023-06-08T03:51:55.630 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.626+0000 7f1e9bede700 1 -- start start 2023-06-08T03:51:55.631 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9bede700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1e9406a460 0x7f1e94071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9bede700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1e940cd6c0 0x7f1e94071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9bede700 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940cee20 con 0x7f1e9415f6c0 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9bede700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e940cf330 con 0x7f1e9406a460 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9bede700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9406fa00 con 0x7f1e940cd6c0 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1e9406a460 0x7f1e94071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1e9406a460 0x7f1e94071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37748/0 (socket says 172.21.15.79:37748) 2023-06-08T03:51:55.632 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99c7a700 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1e9415f6c0 0x7f1e94069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:32990/0 (socket says 172.21.15.79:32990) 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 -- 172.21.15.79:0/3969135286 learned_addr learned my addr 172.21.15.79:0/3969135286 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9a47b700 1 --2- 172.21.15.79:0/3969135286 >> v2:172.21.15.156:3300/0 conn(0x7f1e940cd6c0 0x7f1e94071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 -- 172.21.15.79:0/3969135286 >> v2:172.21.15.156:3300/0 conn(0x7f1e940cd6c0 msgr2=0x7f1e94071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 --2- 172.21.15.79:0/3969135286 >> v2:172.21.15.156:3300/0 conn(0x7f1e940cd6c0 0x7f1e94071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 -- 172.21.15.79:0/3969135286 >> v1:172.21.15.79:6789/0 conn(0x7f1e9415f6c0 legacy=0x7f1e94069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 -- 172.21.15.79:0/3969135286 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940cd1e0 con 0x7f1e9406a460 2023-06-08T03:51:55.633 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e9a47b700 1 --2- 172.21.15.79:0/3969135286 >> v2:172.21.15.156:3300/0 conn(0x7f1e940cd6c0 0x7f1e94071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-08T03:51:55.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e99479700 1 --2- 172.21.15.79:0/3969135286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e94071050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1e940aecb0 tx=0x7f1e84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16e086b982034c3 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e98c78700 1 -- 172.21.15.79:0/3969135286 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400b870 con 0x7f1e9406a460 2023-06-08T03:51:55.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.627+0000 7f1e98c78700 1 -- 172.21.15.79:0/3969135286 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8400b9d0 con 0x7f1e9406a460 2023-06-08T03:51:55.634 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e98c78700 1 -- 172.21.15.79:0/3969135286 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8400bd60 con 0x7f1e9406a460 2023-06-08T03:51:55.636 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 -- 172.21.15.79:0/3969135286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 msgr2=0x7f1e94071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 --2- 172.21.15.79:0/3969135286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e94071050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1e940aecb0 tx=0x7f1e84009bd0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 -- 172.21.15.79:0/3969135286 shutdown_connections 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 --2- 172.21.15.79:0/3969135286 >> v2:172.21.15.156:3300/0 conn(0x7f1e940cd6c0 0x7f1e94071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 --2- 172.21.15.79:0/3969135286 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e94071050 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 -- 172.21.15.79:0/3969135286 >> 172.21.15.79:0/3969135286 conn(0x7f1e940b1d20 msgr2=0x7f1e9406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 -- 172.21.15.79:0/3969135286 shutdown_connections 2023-06-08T03:51:55.637 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.628+0000 7f1e9bede700 1 -- 172.21.15.79:0/3969135286 wait complete. 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.629+0000 7f1e9bede700 1 Processor -- start 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.629+0000 7f1e9bede700 1 -- start start 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9bede700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e941e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9bede700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e940cd6c0 0x7f1e94159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9bede700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9415f6c0 0x7f1e940ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9bede700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940ad6c0 con 0x7f1e9406a460 2023-06-08T03:51:55.638 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9bede700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940ad230 con 0x7f1e9415f6c0 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9bede700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940ad910 con 0x7f1e940cd6c0 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e941e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e941e7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37752/0 (socket says 172.21.15.79:37752) 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 -- 172.21.15.79:0/1125324814 learned_addr learned my addr 172.21.15.79:0/1125324814 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99479700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e940cd6c0 0x7f1e94159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9a47b700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9415f6c0 0x7f1e940ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 -- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e940cd6c0 msgr2=0x7f1e94159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.639 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e940cd6c0 0x7f1e94159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 -- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9415f6c0 msgr2=0x7f1e940ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9415f6c0 0x7f1e940ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e99c7a700 1 -- 172.21.15.79:0/1125324814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84009550 con 0x7f1e9406a460 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.630+0000 7f1e9a47b700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9415f6c0 0x7f1e940ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.631+0000 7f1e99c7a700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e941e7080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000c520 tx=0x7f1e9000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.631+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90005cb0 con 0x7f1e9406a460 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.631+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90005e10 con 0x7f1e9406a460 2023-06-08T03:51:55.640 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.631+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e940adbf0 con 0x7f1e9406a460 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.631+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900072d0 con 0x7f1e9406a460 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.631+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9415ad40 con 0x7f1e9406a460 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.632+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e940ae110 con 0x7f1e9406a460 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.632+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e90007430 con 0x7f1e9406a460 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.632+0000 7f1e8a7fc700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e7806ace0 0x7f1e7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.633+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1e90088860 con 0x7f1e9406a460 2023-06-08T03:51:55.641 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.633+0000 7f1e99479700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e7806ace0 0x7f1e7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:51:55.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.633+0000 7f1e99479700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e7806ace0 0x7f1e7806d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e84014040 tx=0x7f1e84008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:51:55.642 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.637+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e940ae110 con 0x7f1e9406a460 2023-06-08T03:51:55.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.865+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1e940ae110 con 0x7f1e9406a460 2023-06-08T03:51:55.869 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.865+0000 7f1e8a7fc700 1 -- 172.21.15.79:0/1125324814 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1e90017060 con 0x7f1e9406a460 2023-06-08T03:51:55.870 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-08T03:51:55.871 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e7806ace0 msgr2=0x7f1e7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e7806ace0 0x7f1e7806d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1e84014040 tx=0x7f1e84008030 comp rx=0 tx=0).stop 2023-06-08T03:51:55.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 msgr2=0x7f1e941e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:51:55.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e941e7080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000c520 tx=0x7f1e9000c830 comp rx=0 tx=0).stop 2023-06-08T03:51:55.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 shutdown_connections 2023-06-08T03:51:55.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e9415f6c0 0x7f1e940ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.872 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e9406a460 0x7f1e941e7080 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f1e7806ace0 0x7f1e7806d190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.867+0000 7f1e9bede700 1 --2- 172.21.15.79:0/1125324814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e940cd6c0 0x7f1e94159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:51:55.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.868+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 >> 172.21.15.79:0/1125324814 conn(0x7f1e940b1d20 msgr2=0x7f1e9406eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:51:55.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.868+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 shutdown_connections 2023-06-08T03:51:55.873 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:51:55.868+0000 7f1e9bede700 1 -- 172.21.15.79:0/1125324814 wait complete. 2023-06-08T03:51:55.883 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-08T03:51:55.884 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-08T03:51:55.884 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2023-06-08T03:51:55.884 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-08T03:51:55.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:55.885 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-08T03:51:55.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-08T03:51:55.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-08T03:51:55.885 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-08T03:51:55.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:55.886 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2023-06-08T03:51:55.903 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-08T03:51:55.903 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:55.904 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42271sec preferred_lft 42271sec 2023-06-08T03:51:55.905 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::6d1d:ad0f:719a:51d1/64 scope link noprefixroute 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 16:7f:25:be:e6:a2 brd ff:ff:ff:ff:ff:ff 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::7883:54ff:fea8:cae8/64 scope link 2023-06-08T03:51:55.906 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 16:7f:25:be:e6:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::147f:25ff:febe:e6a2/64 scope link 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether d6:45:69:ed:11:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::d445:69ff:feed:1152/64 scope link 2023-06-08T03:51:55.907 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2023-06-08T03:51:55.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:55.908 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-08T03:51:55.968 INFO:teuthology.orchestra.run.smithi079.stdout:1 2023-06-08T03:51:55.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:55.970 DEBUG:teuthology.orchestra.run.smithi079:> route 2023-06-08T03:51:56.028 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2023-06-08T03:51:56.028 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-08T03:51:56.028 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-08T03:51:56.029 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-08T03:51:56.029 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-08T03:51:56.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:56.029 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec bash -c ' 2023-06-08T03:51:56.030 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2023-06-08T03:51:56.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-08T03:51:56.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-08T03:51:56.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-08T03:51:56.031 DEBUG:teuthology.orchestra.run.smithi079:> ') 2023-06-08T03:51:56.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:56.134 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2023-06-08T03:51:56.150 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-08T03:51:56.150 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-08T03:51:56.151 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-08T03:51:56.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:56.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2023-06-08T03:51:56.216 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-08T03:51:56.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:51:56.217 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:56.272 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-08T03:51:56.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-08T03:51:56.273 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:56.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:51:56.395 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:51:56.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:51:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:51:56.461 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:51:56.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:56.462 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:51:56.518 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-08T03:51:56.518 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-08T03:51:56.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2023-06-08T03:51:56.586 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-08T03:51:56.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-08T03:51:56.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:2023-06-08T03:51:56.642+0000 7f969600d5c0 -1 init, newargv = 0x56059a9ffab0 newargc=15 2023-06-08T03:51:56.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[118834]: starting ceph client 2023-06-08T03:51:56.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[118834]: starting fuse 2023-06-08T03:51:56.651 INFO:teuthology.orchestra.run.smithi079.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-08T03:51:56.653 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-08T03:51:56.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:56.654 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2023-06-08T03:51:56.710 INFO:teuthology.orchestra.run.smithi079.stdout:47 2023-06-08T03:51:56.710 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-06-08T03:51:56.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-08T03:51:56.711 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2023-06-08T03:51:56.712 DEBUG:teuthology.orchestra.run.smithi079:> import re 2023-06-08T03:51:56.712 DEBUG:teuthology.orchestra.run.smithi079:> import os 2023-06-08T03:51:56.712 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2023-06-08T03:51:56.713 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:51:56.713 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2023-06-08T03:51:56.713 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-08T03:51:56.713 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2023-06-08T03:51:56.714 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-08T03:51:56.714 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:51:56.714 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2023-06-08T03:51:56.715 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2023-06-08T03:51:56.715 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2023-06-08T03:51:56.715 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2023-06-08T03:51:56.716 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:51:56.716 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2023-06-08T03:51:56.716 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-08T03:51:56.716 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-08T03:51:56.717 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-08T03:51:56.717 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2023-06-08T03:51:56.718 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2023-06-08T03:51:56.718 DEBUG:teuthology.orchestra.run.smithi079:> return f 2023-06-08T03:51:56.718 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-08T03:51:56.719 DEBUG:teuthology.orchestra.run.smithi079:> 2023-06-08T03:51:56.719 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.0")) 2023-06-08T03:51:56.719 DEBUG:teuthology.orchestra.run.smithi079:> ' 2023-06-08T03:51:56.831 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.0.118834.asok 2023-06-08T03:51:57.014 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.118834.asok 2023-06-08T03:51:57.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:57.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.118834.asok status 2023-06-08T03:51:57.169 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2023-06-08T03:51:57.169 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2023-06-08T03:51:57.169 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "a02638f808b8fe11bccdd01222c0e6e2eb71120b", 2023-06-08T03:51:57.169 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev)", 2023-06-08T03:51:57.169 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "0", 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "118834", 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2023-06-08T03:51:57.170 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 5790, 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 5790 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:57.171 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 281848215 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2023-06-08T03:51:57.172 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 281848215 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.5790 192.168.0.1:0/281848215", 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.1:0/281848215", 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 17, 2023-06-08T03:51:57.173 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 38, 2023-06-08T03:51:57.174 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2023-06-08T03:51:57.174 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2023-06-08T03:51:57.174 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2023-06-08T03:51:57.174 INFO:teuthology.orchestra.run.smithi079.stdout:} 2023-06-08T03:51:57.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:57.179 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2023-06-08T03:51:57.179 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:57.196 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2023-06-08T03:51:57.196 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:57.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:57.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-08T03:51:57.262 DEBUG:teuthology.orchestra.run.smithi079:> mktemp -d --suffix=xfstests-dev 2023-06-08T03:51:57.318 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.l6an1raz8Jxfstests-dev 2023-06-08T03:51:57.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:57.318 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.l6an1raz8Jxfstests-dev) 2023-06-08T03:51:57.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:57.385 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.l6an1raz8Jxfstests-dev) 2023-06-08T03:51:57.441 INFO:teuthology.orchestra.run.smithi079.stderr:Cloning into '/tmp/tmp.l6an1raz8Jxfstests-dev'... 2023-06-08T03:51:58.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:58.068 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-06-08T03:51:58.128 DEBUG:teuthology.orchestra.run.smithi079:> mktemp -d --suffix=test 2023-06-08T03:51:58.185 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.umJjrxL0owtest 2023-06-08T03:51:58.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:51:58.185 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-06-08T03:51:58.243 DEBUG:teuthology.orchestra.run.smithi079:> mktemp -d --suffix=scratch 2023-06-08T03:51:58.298 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.0RMespqXl6scratch 2023-06-08T03:51:58.299 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "rhel" and "8.6". 2023-06-08T03:51:58.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python2 sqlite xfsprogs-qa-devel 2023-06-08T03:51:58.692 INFO:teuthology.orchestra.run.smithi079.stdout:Updating Subscription Management repositories. 2023-06-08T03:51:59.517 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:05:10 ago on Thu 08 Jun 2023 03:46:49 AM UTC. 2023-06-08T03:52:00.790 INFO:teuthology.orchestra.run.smithi079.stdout:Package acl-2.2.53-1.el8.x86_64 is already installed. 2023-06-08T03:52:00.791 INFO:teuthology.orchestra.run.smithi079.stdout:Package attr-2.4.48-3.el8.x86_64 is already installed. 2023-06-08T03:52:00.792 INFO:teuthology.orchestra.run.smithi079.stdout:Package automake-1.16.1-7.el8.noarch is already installed. 2023-06-08T03:52:00.793 INFO:teuthology.orchestra.run.smithi079.stdout:Package bc-1.07.1-5.el8.x86_64 is already installed. 2023-06-08T03:52:00.794 INFO:teuthology.orchestra.run.smithi079.stdout:Package dbench-4.0-20.el8.x86_64 is already installed. 2023-06-08T03:52:00.796 INFO:teuthology.orchestra.run.smithi079.stdout:Package e2fsprogs-1.45.6-4.el8.x86_64 is already installed. 2023-06-08T03:52:00.798 INFO:teuthology.orchestra.run.smithi079.stdout:Package gawk-4.2.1-4.el8.x86_64 is already installed. 2023-06-08T03:52:00.800 INFO:teuthology.orchestra.run.smithi079.stdout:Package gcc-8.5.0-10.1.el8_6.x86_64 is already installed. 2023-06-08T03:52:00.802 INFO:teuthology.orchestra.run.smithi079.stdout:Package libtool-2.4.6-25.el8.x86_64 is already installed. 2023-06-08T03:52:00.803 INFO:teuthology.orchestra.run.smithi079.stdout:Package lvm2-8:2.03.14-3.el8_6.2.x86_64 is already installed. 2023-06-08T03:52:00.804 INFO:teuthology.orchestra.run.smithi079.stdout:Package make-1:4.2.1-11.el8.x86_64 is already installed. 2023-06-08T03:52:00.805 INFO:teuthology.orchestra.run.smithi079.stdout:Package psmisc-23.1-5.el8.x86_64 is already installed. 2023-06-08T03:52:00.806 INFO:teuthology.orchestra.run.smithi079.stdout:Package quota-1:4.04-14.el8.x86_64 is already installed. 2023-06-08T03:52:00.807 INFO:teuthology.orchestra.run.smithi079.stdout:Package sed-4.5-5.el8.x86_64 is already installed. 2023-06-08T03:52:00.808 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-08T03:52:00.809 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-08T03:52:00.810 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-08T03:52:00.811 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-08T03:52:00.812 INFO:teuthology.orchestra.run.smithi079.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2023-06-08T03:52:00.814 INFO:teuthology.orchestra.run.smithi079.stdout:Package libuuid-devel-2.32.1-35.el8.x86_64 is already installed. 2023-06-08T03:52:00.815 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-devel-5.0.0-10.el8.x86_64 is already installed. 2023-06-08T03:52:00.817 INFO:teuthology.orchestra.run.smithi079.stdout:Package sqlite-3.26.0-15.el8.x86_64 is already installed. 2023-06-08T03:52:00.821 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-08T03:52:00.934 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2023-06-08T03:52:00.937 INFO:teuthology.orchestra.run.smithi079.stdout:==================================================================================================================== 2023-06-08T03:52:00.937 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2023-06-08T03:52:00.937 INFO:teuthology.orchestra.run.smithi079.stdout:==================================================================================================================== 2023-06-08T03:52:00.937 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2023-06-08T03:52:00.937 INFO:teuthology.orchestra.run.smithi079.stdout: dump x86_64 1:0.4-0.36.b46.el8 rhel-8-for-x86_64-baseos-rpms 164 k 2023-06-08T03:52:00.937 INFO:teuthology.orchestra.run.smithi079.stdout: fio x86_64 3.19-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout: indent x86_64 2.2.13-2.el8 epel 187 k 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout: python2 x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout: daxctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 42 k 2023-06-08T03:52:00.938 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemblk x86_64 1.6.1-1.el8 rhel-8-for-x86_64-appstream-rpms 92 k 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: ndctl-libs x86_64 71.1-3.el8 rhel-8-for-x86_64-baseos-rpms 79 k 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: python2-libs x86_64 2.7.18-10.module+el8.6.0+14191+7fdd52cd rhel-8-for-x86_64-appstream-rpms 6.0 M 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: python2-pip-wheel noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 892 k 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: python2-setuptools-wheel noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 287 k 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: python2-wheel-wheel noarch 1:0.31.1-3.module+el8.5.0+12203+77770ab7 rhel-8-for-x86_64-appstream-rpms 45 k 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: rmt x86_64 2:1.5.3-13.el8 rhel-8-for-x86_64-baseos-rpms 51 k 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2023-06-08T03:52:00.939 INFO:teuthology.orchestra.run.smithi079.stdout: python2-pip noarch 9.0.3-19.module+el8.6.0+13001+ad200bd9 rhel-8-for-x86_64-appstream-rpms 1.6 M 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout: python2-setuptools noarch 39.0.1-13.module+el8.4.0+9442+27d0e81c rhel-8-for-x86_64-appstream-rpms 642 k 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout:Enabling module streams: 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout: python27 2.7 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout:==================================================================================================================== 2023-06-08T03:52:00.940 INFO:teuthology.orchestra.run.smithi079.stdout:Install 14 Packages 2023-06-08T03:52:00.941 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 2 Packages 2023-06-08T03:52:00.941 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:52:00.941 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 12 M 2023-06-08T03:52:00.941 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2023-06-08T03:52:01.167 INFO:teuthology.orchestra.run.smithi079.stdout:(1/16): dump-0.4-0.36.b46.el8.x86_64.rpm 2.3 MB/s | 164 kB 00:00 2023-06-08T03:52:01.192 INFO:teuthology.orchestra.run.smithi079.stdout:(2/16): rmt-1.5.3-13.el8.x86_64.rpm 549 kB/s | 51 kB 00:00 2023-06-08T03:52:01.209 INFO:teuthology.orchestra.run.smithi079.stdout:(3/16): daxctl-libs-71.1-3.el8.x86_64.rpm 993 kB/s | 42 kB 00:00 2023-06-08T03:52:01.234 INFO:teuthology.orchestra.run.smithi079.stdout:(4/16): ndctl-libs-71.1-3.el8.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2023-06-08T03:52:01.260 INFO:teuthology.orchestra.run.smithi079.stdout:(5/16): python2-setuptools-wheel-39.0.1-13.modu 5.6 MB/s | 287 kB 00:00 2023-06-08T03:52:01.301 INFO:teuthology.orchestra.run.smithi079.stdout:(6/16): python2-setuptools-39.0.1-13.module+el8 9.4 MB/s | 642 kB 00:00 2023-06-08T03:52:01.327 INFO:teuthology.orchestra.run.smithi079.stdout:(7/16): indent-2.2.13-2.el8.x86_64.rpm 815 kB/s | 187 kB 00:00 2023-06-08T03:52:01.344 INFO:teuthology.orchestra.run.smithi079.stdout:(8/16): python2-wheel-wheel-0.31.1-3.module+el8 532 kB/s | 45 kB 00:00 2023-06-08T03:52:01.377 INFO:teuthology.orchestra.run.smithi079.stdout:(9/16): fio-3.19-3.el8.x86_64.rpm 6.9 MB/s | 538 kB 00:00 2023-06-08T03:52:01.403 INFO:teuthology.orchestra.run.smithi079.stdout:(10/16): libpmemblk-1.6.1-1.el8.x86_64.rpm 1.2 MB/s | 92 kB 00:00 2023-06-08T03:52:01.428 INFO:teuthology.orchestra.run.smithi079.stdout:(11/16): python2-2.7.18-10.module+el8.6.0+14191 1.4 MB/s | 110 kB 00:00 2023-06-08T03:52:01.478 INFO:teuthology.orchestra.run.smithi079.stdout:(12/16): python2-pip-9.0.3-19.module+el8.6.0+13 22 MB/s | 1.6 MB 00:00 2023-06-08T03:52:01.512 INFO:teuthology.orchestra.run.smithi079.stdout:(13/16): python2-pip-wheel-9.0.3-19.module+el8. 10 MB/s | 892 kB 00:00 2023-06-08T03:52:01.545 INFO:teuthology.orchestra.run.smithi079.stdout:(14/16): sqlite-3.26.0-16.el8_6.x86_64.rpm 9.8 MB/s | 668 kB 00:00 2023-06-08T03:52:01.579 INFO:teuthology.orchestra.run.smithi079.stdout:(15/16): sqlite-libs-3.26.0-16.el8_6.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2023-06-08T03:52:01.663 INFO:teuthology.orchestra.run.smithi079.stdout:(16/16): python2-libs-2.7.18-10.module+el8.6.0+ 21 MB/s | 6.0 MB 00:00 2023-06-08T03:52:01.663 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2023-06-08T03:52:01.664 INFO:teuthology.orchestra.run.smithi079.stdout:Total 16 MB/s | 12 MB 00:00 2023-06-08T03:52:01.866 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2023-06-08T03:52:01.906 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2023-06-08T03:52:01.906 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2023-06-08T03:52:02.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:52:02.075 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:52:02.090 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2023-06-08T03:52:02.096 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2023-06-08T03:52:02.807 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2023-06-08T03:52:02.995 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 1/18 2023-06-08T03:52:03.131 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : daxctl-libs-71.1-3.el8.x86_64 2/18 2023-06-08T03:52:03.208 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: daxctl-libs-71.1-3.el8.x86_64 2/18 2023-06-08T03:52:03.439 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ndctl-libs-71.1-3.el8.x86_64 3/18 2023-06-08T03:52:03.517 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ndctl-libs-71.1-3.el8.x86_64 3/18 2023-06-08T03:52:03.689 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-06-08T03:52:03.774 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libpmemblk-1.6.1-1.el8.x86_64 4/18 2023-06-08T03:52:03.926 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 5/18 2023-06-08T03:52:04.202 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 6/18 2023-06-08T03:52:04.850 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 7/18 2023-06-08T03:52:05.091 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 8/18 2023-06-08T03:52:05.270 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 9/18 2023-06-08T03:52:06.378 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 9/18 2023-06-08T03:52:06.711 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 10/18 2023-06-08T03:52:06.889 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rmt-2:1.5.3-13.el8.x86_64 11/18 2023-06-08T03:52:07.181 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : dump-1:0.4-0.36.b46.el8.x86_64 12/18 2023-06-08T03:52:07.435 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fio-3.19-3.el8.x86_64 13/18 2023-06-08T03:52:07.609 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 14/18 2023-06-08T03:52:07.766 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 15/18 2023-06-08T03:52:07.964 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : indent-2.2.13-2.el8.x86_64 16/18 2023-06-08T03:52:08.097 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 17/18 2023-06-08T03:52:08.244 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-06-08T03:52:08.520 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-06-08T03:52:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : indent-2.2.13-2.el8.x86_64 1/18 2023-06-08T03:52:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : dump-1:0.4-0.36.b46.el8.x86_64 2/18 2023-06-08T03:52:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rmt-2:1.5.3-13.el8.x86_64 3/18 2023-06-08T03:52:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : daxctl-libs-71.1-3.el8.x86_64 4/18 2023-06-08T03:52:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ndctl-libs-71.1-3.el8.x86_64 5/18 2023-06-08T03:52:08.521 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-setuptools-wheel-39.0.1-13.module+el8.4.0+ 6/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-setuptools-39.0.1-13.module+el8.4.0+9442+2 7/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+1220 8/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fio-3.19-3.el8.x86_64 9/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemblk-1.6.1-1.el8.x86_64 10/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x8 11/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52 12/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9 13/18 2023-06-08T03:52:08.522 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad 14/18 2023-06-08T03:52:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 15/18 2023-06-08T03:52:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 16/18 2023-06-08T03:52:08.523 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 17/18 2023-06-08T03:52:10.027 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 18/18 2023-06-08T03:52:10.027 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:05:21 ago on Thu 08 Jun 2023 03:46:49 AM UTC. 2023-06-08T03:52:12.447 INFO:teuthology.orchestra.run.smithi079.stdout:Installed products updated. 2023-06-08T03:52:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:52:12.639 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2023-06-08T03:52:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: sqlite-3.26.0-16.el8_6.x86_64 sqlite-libs-3.26.0-16.el8_6.x86_64 2023-06-08T03:52:12.639 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2023-06-08T03:52:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: daxctl-libs-71.1-3.el8.x86_64 2023-06-08T03:52:12.639 INFO:teuthology.orchestra.run.smithi079.stdout: dump-1:0.4-0.36.b46.el8.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: fio-3.19-3.el8.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: indent-2.2.13-2.el8.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemblk-1.6.1-1.el8.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: ndctl-libs-71.1-3.el8.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: python2-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: python2-libs-2.7.18-10.module+el8.6.0+14191+7fdd52cd.x86_64 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: python2-pip-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-06-08T03:52:12.640 INFO:teuthology.orchestra.run.smithi079.stdout: python2-pip-wheel-9.0.3-19.module+el8.6.0+13001+ad200bd9.noarch 2023-06-08T03:52:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: python2-setuptools-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-06-08T03:52:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: python2-setuptools-wheel-39.0.1-13.module+el8.4.0+9442+27d0e81c.noarch 2023-06-08T03:52:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: python2-wheel-wheel-1:0.31.1-3.module+el8.5.0+12203+77770ab7.noarch 2023-06-08T03:52:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: rmt-2:1.5.3-13.el8.x86_64 2023-06-08T03:52:12.641 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:52:12.641 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2023-06-08T03:52:12.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo useradd -m fsgqa 2023-06-08T03:52:12.826 INFO:teuthology.orchestra.run.smithi079.stderr:useradd: user 'fsgqa' already exists 2023-06-08T03:52:12.827 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-06-08T03:52:12.827 DEBUG:teuthology.orchestra.run.smithi079:> sudo groupadd fsgqa 2023-06-08T03:52:12.893 INFO:teuthology.orchestra.run.smithi079.stderr:groupadd: group 'fsgqa' already exists 2023-06-08T03:52:12.894 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-06-08T03:52:12.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo useradd fsgqa2 2023-06-08T03:52:13.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo useradd 123456-fsgqa 2023-06-08T03:52:14.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-06-08T03:52:14.933 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.929+0000 7f857a6d9700 1 Processor -- start 2023-06-08T03:52:14.934 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 -- start start 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f8574158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8574158f80 0x7f85740ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f85740cd9b0 0x7f8574159c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8574159a10 con 0x7f8574158f80 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85740cc810 con 0x7f85740cd9b0 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f857a6d9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8574159450 con 0x7f85740c9630 2023-06-08T03:52:14.936 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.930+0000 7f85737fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8574158f80 0x7f85740ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8574158f80 0x7f85740ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37794/0 (socket says 172.21.15.79:37794) 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 -- 172.21.15.79:0/404796547 learned_addr learned my addr 172.21.15.79:0/404796547 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f8578c76700 1 --2- 172.21.15.79:0/404796547 >> v2:172.21.15.156:3300/0 conn(0x7f85740cd9b0 0x7f8574159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f8573fff700 1 --2- 172.21.15.79:0/404796547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f8574158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 -- 172.21.15.79:0/404796547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 msgr2=0x7f8574158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 --2- 172.21.15.79:0/404796547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f8574158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.937 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 -- 172.21.15.79:0/404796547 >> v2:172.21.15.156:3300/0 conn(0x7f85740cd9b0 msgr2=0x7f8574159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:52:14.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 --2- 172.21.15.79:0/404796547 >> v2:172.21.15.156:3300/0 conn(0x7f85740cd9b0 0x7f8574159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 -- 172.21.15.79:0/404796547 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574159730 con 0x7f8574158f80 2023-06-08T03:52:14.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f85737fe700 1 --2- 172.21.15.79:0/404796547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740ccca0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f85740cf3a0 tx=0x7f8568009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63f0a7030940246 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:52:14.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f8572ffd700 1 -- 172.21.15.79:0/404796547 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800b870 con 0x7f8574158f80 2023-06-08T03:52:14.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f8572ffd700 1 -- 172.21.15.79:0/404796547 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856800b9d0 con 0x7f8574158f80 2023-06-08T03:52:14.938 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.931+0000 7f8572ffd700 1 -- 172.21.15.79:0/404796547 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800bd60 con 0x7f8574158f80 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 -- 172.21.15.79:0/404796547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 msgr2=0x7f85740ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 --2- 172.21.15.79:0/404796547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740ccca0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f85740cf3a0 tx=0x7f8568009bd0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 -- 172.21.15.79:0/404796547 shutdown_connections 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 --2- 172.21.15.79:0/404796547 >> v2:172.21.15.156:3300/0 conn(0x7f85740cd9b0 0x7f8574159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 --2- 172.21.15.79:0/404796547 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f8574158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 --2- 172.21.15.79:0/404796547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740ccca0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.939 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 -- 172.21.15.79:0/404796547 >> 172.21.15.79:0/404796547 conn(0x7f8574069d50 msgr2=0x7f857406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:52:14.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 -- 172.21.15.79:0/404796547 shutdown_connections 2023-06-08T03:52:14.940 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.932+0000 7f857a6d9700 1 -- 172.21.15.79:0/404796547 wait complete. 2023-06-08T03:52:14.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.933+0000 7f857a6d9700 1 Processor -- start 2023-06-08T03:52:14.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.933+0000 7f857a6d9700 1 -- start start 2023-06-08T03:52:14.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f857a6d9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f85741f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f857a6d9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85740cd9b0 0x7f857414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f857a6d9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.942 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f857a6d9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85741f3250 con 0x7f8574158f80 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f857a6d9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85740c6790 con 0x7f85740cd9b0 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f857a6d9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85740c6a70 con 0x7f85740c9630 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37802/0 (socket says 172.21.15.79:37802) 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 -- 172.21.15.79:0/1210679532 learned_addr learned my addr 172.21.15.79:0/1210679532 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f85737fe700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85740cd9b0 0x7f857414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.943 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8573fff700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f85741f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 -- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 msgr2=0x7f85741f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f85741f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 -- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85740cd9b0 msgr2=0x7f857414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85740cd9b0 0x7f857414e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 -- 172.21.15.79:0/1210679532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8568009550 con 0x7f8574158f80 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8573fff700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f85741f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:52:14.944 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.934+0000 7f8578c76700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740c5d10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f856c007e30 tx=0x7f856c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.935+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c00ede0 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.935+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c014420 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.935+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85741f34a0 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.935+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c0147d0 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.935+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857414ee30 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.935+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85741f39c0 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.937+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f856c014930 con 0x7f8574158f80 2023-06-08T03:52:14.945 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.937+0000 7f8570ff9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f856006ace0 0x7f856006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:14.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.937+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f856c08a610 con 0x7f8574158f80 2023-06-08T03:52:14.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.937+0000 7f8573fff700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f856006ace0 0x7f856006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:14.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.938+0000 7f8573fff700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f856006ace0 0x7f856006d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8564009970 tx=0x7f8564008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:52:14.946 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:14.940+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f856c0555d0 con 0x7f8574158f80 2023-06-08T03:52:15.219 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.215+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f85741f39c0 con 0x7f8574158f80 2023-06-08T03:52:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.216+0000 7f8570ff9700 1 -- 172.21.15.79:0/1210679532 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f85741f39c0 con 0x7f8574158f80 2023-06-08T03:52:15.220 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:52:15.220 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":1,"fsid":"141881e0-5474-41d9-8205-1a44c31c5e0b","modified":"2023-06-08T03:49:33.594151Z","created":"2023-06-08T03:49:33.594151Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3301","nonce":0},{"type":"v1","addr":"172.21.15.156:6790","nonce":0}]},"addr":"172.21.15.156:6790/0","public_addr":"172.21.15.156:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-06-08T03:52:15.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f856006ace0 msgr2=0x7f856006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f856006ace0 0x7f856006d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8564009970 tx=0x7f8564008040 comp rx=0 tx=0).stop 2023-06-08T03:52:15.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 msgr2=0x7f85740c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740c5d10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f856c007e30 tx=0x7f856c00e3c0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.222 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 shutdown_connections 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8574158f80 0x7f85740c5d10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f856006ace0 0x7f856006d190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f85740c9630 0x7f85741f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 --2- 172.21.15.79:0/1210679532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85740cd9b0 0x7f857414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 >> 172.21.15.79:0/1210679532 conn(0x7f8574069d50 msgr2=0x7f85740b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 shutdown_connections 2023-06-08T03:52:15.223 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.218+0000 7f857a6d9700 1 -- 172.21.15.79:0/1210679532 wait complete. 2023-06-08T03:52:15.224 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 1 2023-06-08T03:52:15.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2023-06-08T03:52:15.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.370+0000 7f15023f9700 1 Processor -- start 2023-06-08T03:52:15.374 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.370+0000 7f15023f9700 1 -- start start 2023-06-08T03:52:15.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f15023f9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:15.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f15023f9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14fc0cd000 0x7f14fc0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:15.375 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f15023f9700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc15d930 con 0x7f14fc0b20f0 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f15023f9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14fc15e770 con 0x7f14fc0cd000 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f15023f9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0cd9b0 con 0x7f14fc0d0240 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f14fbfff700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f14fb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14fc0cd000 0x7f14fc0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f1500996700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f14fc0b20f0 0x7f14fc0b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.79:35216/0 (socket says 172.21.15.79:35216) 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f14fb7fe700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f14fc0cd000 0x7f14fc0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.79:60702/0 (socket says 172.21.15.79:60702) 2023-06-08T03:52:15.376 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f1500996700 1 -- 172.21.15.79:0/2489395696 learned_addr learned my addr 172.21.15.79:0/2489395696 (peer_addr_for_me v1:172.21.15.79:0/0) 2023-06-08T03:52:15.377 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.371+0000 7f14fbfff700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc0af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.79:52128/0 (socket says 172.21.15.79:52128) 2023-06-08T03:52:15.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14faffd700 1 -- 172.21.15.79:0/2489395696 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496051095 0 0) 0x7f14fc15d930 con 0x7f14fc0b20f0 2023-06-08T03:52:15.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14faffd700 1 -- 172.21.15.79:0/2489395696 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e40035b0 con 0x7f14fc0b20f0 2023-06-08T03:52:15.378 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14fb7fe700 1 -- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 msgr2=0x7f14fc0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14fb7fe700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14fb7fe700 1 -- 172.21.15.79:0/2489395696 >> v1:172.21.15.156:6789/0 conn(0x7f14fc0b20f0 legacy=0x7f14fc0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14fb7fe700 1 -- 172.21.15.79:0/2489395696 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc15db60 con 0x7f14fc0cd000 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14fbfff700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14fb7fe700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc0cd470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009840 tx=0x7f14f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4b1b4cfcb1ad4d server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14faffd700 1 -- 172.21.15.79:0/2489395696 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0011830 con 0x7f14fc0cd000 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14faffd700 1 -- 172.21.15.79:0/2489395696 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0011990 con 0x7f14fc0cd000 2023-06-08T03:52:15.379 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.372+0000 7f14faffd700 1 -- 172.21.15.79:0/2489395696 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0011d20 con 0x7f14fc0cd000 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 -- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 msgr2=0x7f14fc0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc0cd470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009840 tx=0x7f14f0009b50 comp rx=0 tx=0).stop 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 -- 172.21.15.79:0/2489395696 shutdown_connections 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 --2- 172.21.15.79:0/2489395696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc0cd470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 -- 172.21.15.79:0/2489395696 >> 172.21.15.79:0/2489395696 conn(0x7f14fc06e1e0 msgr2=0x7f14fc06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:52:15.380 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 -- 172.21.15.79:0/2489395696 shutdown_connections 2023-06-08T03:52:15.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.373+0000 7f15023f9700 1 -- 172.21.15.79:0/2489395696 wait complete. 2023-06-08T03:52:15.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.374+0000 7f15023f9700 1 Processor -- start 2023-06-08T03:52:15.381 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.374+0000 7f15023f9700 1 -- start start 2023-06-08T03:52:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f15023f9700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 0x7f14fc1ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f15023f9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc06e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f15023f9700 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f15023f9700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0c48e0 con 0x7f14fc0b20f0 2023-06-08T03:52:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f15023f9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc1ef620 con 0x7f14fc0cd000 2023-06-08T03:52:15.383 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f15023f9700 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0ca040 con 0x7f14fc0d0240 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f14fbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 0x7f14fc1ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f14fbfff700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 0x7f14fc1ef0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37812/0 (socket says 172.21.15.79:37812) 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f14fbfff700 1 -- 172.21.15.79:0/1316255349 learned_addr learned my addr 172.21.15.79:0/1316255349 (peer_addr_for_me v2:172.21.15.79:0/0) 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f14fb7fe700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc06e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f1500996700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f14fbfff700 1 -- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 msgr2=0x7f14fc06eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.384 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.375+0000 7f14fbfff700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc06eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14fbfff700 1 -- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 msgr2=0x7f14fc06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14fbfff700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc06e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14fbfff700 1 -- 172.21.15.79:0/1316255349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0094a0 con 0x7f14fc0b20f0 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14fb7fe700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc06e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14fbfff700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 0x7f14fc1ef0e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00ec10 tx=0x7f14ec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00c910 con 0x7f14fc0b20f0 2023-06-08T03:52:15.385 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec00ca70 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f00094a0 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00cde0 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.376+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc0c9680 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.377+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14fc0c4b50 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.377+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14ec013470 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.378+0000 7f14f8ff9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f14e806ace0 0x7f14e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.378+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f14ec089790 con 0x7f14fc0b20f0 2023-06-08T03:52:15.386 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.378+0000 7f14fb7fe700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f14e806ace0 0x7f14e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-08T03:52:15.387 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.378+0000 7f14fb7fe700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f14e806ace0 0x7f14e806d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009fd0 tx=0x7f14f0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-08T03:52:15.389 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.386+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f14ec054670 con 0x7f14fc0b20f0 2023-06-08T03:52:15.651 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.647+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) v1 -- 0x7f14fc073e90 con 0x7f14fc0b20f0 2023-06-08T03:52:15.652 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.648+0000 7f14f8ff9700 1 -- 172.21.15.79:0/1316255349 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v9) v1 ==== 82+0+151 (secure 0 0 0) 0x7f14ec057c90 con 0x7f14fc0b20f0 2023-06-08T03:52:15.652 INFO:teuthology.orchestra.run.smithi079.stdout:[client.admin] 2023-06-08T03:52:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQDNT4FkcImCMBAAXrG1xKI+tIRnnC0V5ZRh7A== 2023-06-08T03:52:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: caps mds = "allow *" 2023-06-08T03:52:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: caps mgr = "allow *" 2023-06-08T03:52:15.652 INFO:teuthology.orchestra.run.smithi079.stdout: caps mon = "allow *" 2023-06-08T03:52:15.653 INFO:teuthology.orchestra.run.smithi079.stdout: caps osd = "allow *" 2023-06-08T03:52:15.653 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f14e806ace0 msgr2=0x7f14e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f14e806ace0 0x7f14e806d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009fd0 tx=0x7f14f0008030 comp rx=0 tx=0).stop 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 msgr2=0x7f14fc1ef0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 0x7f14fc1ef0e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00ec10 tx=0x7f14ec00ef20 comp rx=0 tx=0).stop 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 shutdown_connections 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:6832/2537078349,v1:172.21.15.156:6833/2537078349] conn(0x7f14e806ace0 0x7f14e806d190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f14fc0d0240 0x7f14fc06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.654 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14fc0b20f0 0x7f14fc1ef0e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 --2- 172.21.15.79:0/1316255349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14fc0cd000 0x7f14fc06e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-08T03:52:15.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.649+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 >> 172.21.15.79:0/1316255349 conn(0x7f14fc06e1e0 msgr2=0x7f14fc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-08T03:52:15.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.650+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 shutdown_connections 2023-06-08T03:52:15.655 INFO:teuthology.orchestra.run.smithi079.stderr:2023-06-08T03:52:15.650+0000 7f15023f9700 1 -- 172.21.15.79:0/1316255349 wait complete. 2023-06-08T03:52:15.665 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2023-06-08T03:52:15.681 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.szRy0uNo7x 2023-06-08T03:52:15.682 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:52:15.682 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/tmp/tmp.szRy0uNo7x 2023-06-08T03:52:15.737 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:52:15.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/tmp/tmp.l6an1raz8Jxfstests-dev/local.config 2023-06-08T03:52:15.813 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.umJjrxL0owtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.0RMespqXl6scratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.79:6789 -k /tmp/tmp.szRy0uNo7x --client_mountpoint /test" 2023-06-08T03:52:15.813 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T03:52:15.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/tmp/tmp.l6an1raz8Jxfstests-dev/ceph.exclude 2023-06-08T03:52:15.878 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo make) 2023-06-08T03:52:15.960 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize -cfi 2023-06-08T03:52:16.281 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: putting auxiliary files in '.'. 2023-06-08T03:52:16.281 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file './config.guess' 2023-06-08T03:52:16.284 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file './config.sub' 2023-06-08T03:52:16.287 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file './install-sh' 2023-06-08T03:52:16.291 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file './ltmain.sh' 2023-06-08T03:52:16.311 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-06-08T03:52:16.311 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-06-08T03:52:16.316 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-06-08T03:52:16.320 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-06-08T03:52:16.324 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-06-08T03:52:16.329 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-06-08T03:52:16.334 INFO:teuthology.orchestra.run.smithi079.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-06-08T03:52:16.335 INFO:teuthology.orchestra.run.smithi079.stdout:cp include/install-sh . 2023-06-08T03:52:16.336 INFO:teuthology.orchestra.run.smithi079.stdout:aclocal -I m4 2023-06-08T03:52:17.485 INFO:teuthology.orchestra.run.smithi079.stdout:autoheader 2023-06-08T03:52:17.781 INFO:teuthology.orchestra.run.smithi079.stdout:autoconf 2023-06-08T03:52:17.938 INFO:teuthology.orchestra.run.smithi079.stdout:./configure \ 2023-06-08T03:52:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: --libexecdir=/usr/lib \ 2023-06-08T03:52:17.939 INFO:teuthology.orchestra.run.smithi079.stdout: --exec_prefix=/var/lib 2023-06-08T03:52:18.358 INFO:teuthology.orchestra.run.smithi079.stdout:checking build system type... x86_64-pc-linux-gnu 2023-06-08T03:52:18.358 INFO:teuthology.orchestra.run.smithi079.stdout:checking host system type... x86_64-pc-linux-gnu 2023-06-08T03:52:18.359 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to print strings... printf 2023-06-08T03:52:18.359 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gcc... gcc 2023-06-08T03:52:18.474 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether the C compiler works... yes 2023-06-08T03:52:18.475 INFO:teuthology.orchestra.run.smithi079.stdout:checking for C compiler default output file name... a.out 2023-06-08T03:52:18.500 INFO:teuthology.orchestra.run.smithi079.stdout:checking for suffix of executables... 2023-06-08T03:52:18.548 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether we are cross compiling... no 2023-06-08T03:52:18.565 INFO:teuthology.orchestra.run.smithi079.stdout:checking for suffix of object files... o 2023-06-08T03:52:18.580 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether we are using the GNU C compiler... yes 2023-06-08T03:52:18.596 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether gcc accepts -g... yes 2023-06-08T03:52:18.622 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gcc option to accept ISO C89... none needed 2023-06-08T03:52:18.625 INFO:teuthology.orchestra.run.smithi079.stdout:checking for a sed that does not truncate output... /bin/sed 2023-06-08T03:52:18.627 INFO:teuthology.orchestra.run.smithi079.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-06-08T03:52:18.628 INFO:teuthology.orchestra.run.smithi079.stdout:checking for egrep... /bin/grep -E 2023-06-08T03:52:18.629 INFO:teuthology.orchestra.run.smithi079.stdout:checking for fgrep... /bin/grep -F 2023-06-08T03:52:18.632 INFO:teuthology.orchestra.run.smithi079.stdout:checking for ld used by gcc... /bin/ld 2023-06-08T03:52:18.633 INFO:teuthology.orchestra.run.smithi079.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-06-08T03:52:18.645 INFO:teuthology.orchestra.run.smithi079.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-06-08T03:52:18.661 INFO:teuthology.orchestra.run.smithi079.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-06-08T03:52:18.661 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether ln -s works... yes 2023-06-08T03:52:18.667 INFO:teuthology.orchestra.run.smithi079.stdout:checking the maximum length of command line arguments... 1572864 2023-06-08T03:52:18.669 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-06-08T03:52:18.669 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-06-08T03:52:18.669 INFO:teuthology.orchestra.run.smithi079.stdout:checking for /bin/ld option to reload object files... -r 2023-06-08T03:52:18.670 INFO:teuthology.orchestra.run.smithi079.stdout:checking for objdump... objdump 2023-06-08T03:52:18.670 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to recognize dependent libraries... pass_all 2023-06-08T03:52:18.671 INFO:teuthology.orchestra.run.smithi079.stdout:checking for dlltool... no 2023-06-08T03:52:18.671 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-06-08T03:52:18.671 INFO:teuthology.orchestra.run.smithi079.stdout:checking for ar... ar 2023-06-08T03:52:18.692 INFO:teuthology.orchestra.run.smithi079.stdout:checking for archiver @FILE support... @ 2023-06-08T03:52:18.693 INFO:teuthology.orchestra.run.smithi079.stdout:checking for strip... strip 2023-06-08T03:52:18.693 INFO:teuthology.orchestra.run.smithi079.stdout:checking for ranlib... ranlib 2023-06-08T03:52:18.694 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gawk... gawk 2023-06-08T03:52:18.743 INFO:teuthology.orchestra.run.smithi079.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-06-08T03:52:18.744 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sysroot... no 2023-06-08T03:52:18.746 INFO:teuthology.orchestra.run.smithi079.stdout:checking for a working dd... /bin/dd 2023-06-08T03:52:18.748 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-06-08T03:52:18.761 INFO:teuthology.orchestra.run.smithi079.stdout:checking for mt... no 2023-06-08T03:52:18.763 INFO:teuthology.orchestra.run.smithi079.stdout:checking if : is a manifest tool... no 2023-06-08T03:52:18.804 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to run the C preprocessor... gcc -E 2023-06-08T03:52:18.959 INFO:teuthology.orchestra.run.smithi079.stdout:checking for ANSI C header files... yes 2023-06-08T03:52:18.978 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/types.h... yes 2023-06-08T03:52:19.001 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/stat.h... yes 2023-06-08T03:52:19.024 INFO:teuthology.orchestra.run.smithi079.stdout:checking for stdlib.h... yes 2023-06-08T03:52:19.047 INFO:teuthology.orchestra.run.smithi079.stdout:checking for string.h... yes 2023-06-08T03:52:19.071 INFO:teuthology.orchestra.run.smithi079.stdout:checking for memory.h... yes 2023-06-08T03:52:19.094 INFO:teuthology.orchestra.run.smithi079.stdout:checking for strings.h... yes 2023-06-08T03:52:19.120 INFO:teuthology.orchestra.run.smithi079.stdout:checking for inttypes.h... yes 2023-06-08T03:52:19.143 INFO:teuthology.orchestra.run.smithi079.stdout:checking for stdint.h... yes 2023-06-08T03:52:19.170 INFO:teuthology.orchestra.run.smithi079.stdout:checking for unistd.h... yes 2023-06-08T03:52:19.212 INFO:teuthology.orchestra.run.smithi079.stdout:checking for dlfcn.h... yes 2023-06-08T03:52:19.216 INFO:teuthology.orchestra.run.smithi079.stdout:checking for objdir... .libs 2023-06-08T03:52:19.279 INFO:teuthology.orchestra.run.smithi079.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-06-08T03:52:19.280 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-06-08T03:52:19.294 INFO:teuthology.orchestra.run.smithi079.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-06-08T03:52:19.313 INFO:teuthology.orchestra.run.smithi079.stdout:checking if gcc static flag -static works... no 2023-06-08T03:52:19.333 INFO:teuthology.orchestra.run.smithi079.stdout:checking if gcc supports -c -o file.o... yes 2023-06-08T03:52:19.333 INFO:teuthology.orchestra.run.smithi079.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-06-08T03:52:19.339 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-06-08T03:52:19.363 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether -lc should be explicitly linked in... no 2023-06-08T03:52:19.406 INFO:teuthology.orchestra.run.smithi079.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-06-08T03:52:19.406 INFO:teuthology.orchestra.run.smithi079.stdout:checking how to hardcode library paths into programs... immediate 2023-06-08T03:52:19.415 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether stripping libraries is possible... yes 2023-06-08T03:52:19.416 INFO:teuthology.orchestra.run.smithi079.stdout:checking if libtool supports shared libraries... yes 2023-06-08T03:52:19.416 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether to build shared libraries... yes 2023-06-08T03:52:19.416 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether to build static libraries... yes 2023-06-08T03:52:19.420 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gcc... (cached) gcc 2023-06-08T03:52:19.435 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-06-08T03:52:19.435 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether gcc accepts -g... (cached) yes 2023-06-08T03:52:19.435 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-06-08T03:52:19.435 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gmake... /usr/bin/gmake 2023-06-08T03:52:19.436 INFO:teuthology.orchestra.run.smithi079.stderr:./configure: line 12070: test: $(SHELL): binary operator expected 2023-06-08T03:52:19.437 INFO:teuthology.orchestra.run.smithi079.stdout:checking for tar... /bin/tar 2023-06-08T03:52:19.437 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gzip... /bin/gzip 2023-06-08T03:52:19.443 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether gcc -MM is supported... yes 2023-06-08T03:52:19.443 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sort... /bin/sort 2023-06-08T03:52:19.444 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether ln -s works... yes 2023-06-08T03:52:19.444 INFO:teuthology.orchestra.run.smithi079.stdout:checking for rpm... /bin/rpm 2023-06-08T03:52:19.447 INFO:teuthology.orchestra.run.smithi079.stdout:checking for rpmbuild... no 2023-06-08T03:52:19.472 INFO:teuthology.orchestra.run.smithi079.stdout:checking assert.h usability... yes 2023-06-08T03:52:19.478 INFO:teuthology.orchestra.run.smithi079.stdout:checking assert.h presence... yes 2023-06-08T03:52:19.478 INFO:teuthology.orchestra.run.smithi079.stdout:checking for assert.h... yes 2023-06-08T03:52:19.503 INFO:teuthology.orchestra.run.smithi079.stdout:checking bstring.h usability... no 2023-06-08T03:52:19.513 INFO:teuthology.orchestra.run.smithi079.stdout:checking bstring.h presence... no 2023-06-08T03:52:19.513 INFO:teuthology.orchestra.run.smithi079.stdout:checking for bstring.h... no 2023-06-08T03:52:19.536 INFO:teuthology.orchestra.run.smithi079.stdout:checking libgen.h usability... yes 2023-06-08T03:52:19.543 INFO:teuthology.orchestra.run.smithi079.stdout:checking libgen.h presence... yes 2023-06-08T03:52:19.543 INFO:teuthology.orchestra.run.smithi079.stdout:checking for libgen.h... yes 2023-06-08T03:52:19.569 INFO:teuthology.orchestra.run.smithi079.stdout:checking dirent.h usability... yes 2023-06-08T03:52:19.577 INFO:teuthology.orchestra.run.smithi079.stdout:checking dirent.h presence... yes 2023-06-08T03:52:19.577 INFO:teuthology.orchestra.run.smithi079.stdout:checking for dirent.h... yes 2023-06-08T03:52:19.625 INFO:teuthology.orchestra.run.smithi079.stdout:checking errno.h usability... yes 2023-06-08T03:52:19.632 INFO:teuthology.orchestra.run.smithi079.stdout:checking errno.h presence... yes 2023-06-08T03:52:19.632 INFO:teuthology.orchestra.run.smithi079.stdout:checking for errno.h... yes 2023-06-08T03:52:19.658 INFO:teuthology.orchestra.run.smithi079.stdout:checking malloc.h usability... yes 2023-06-08T03:52:19.666 INFO:teuthology.orchestra.run.smithi079.stdout:checking malloc.h presence... yes 2023-06-08T03:52:19.666 INFO:teuthology.orchestra.run.smithi079.stdout:checking for malloc.h... yes 2023-06-08T03:52:19.692 INFO:teuthology.orchestra.run.smithi079.stdout:checking uuid.h usability... no 2023-06-08T03:52:19.701 INFO:teuthology.orchestra.run.smithi079.stdout:checking uuid.h presence... no 2023-06-08T03:52:19.701 INFO:teuthology.orchestra.run.smithi079.stdout:checking for uuid.h... no 2023-06-08T03:52:19.726 INFO:teuthology.orchestra.run.smithi079.stdout:checking uuid/uuid.h usability... yes 2023-06-08T03:52:19.735 INFO:teuthology.orchestra.run.smithi079.stdout:checking uuid/uuid.h presence... yes 2023-06-08T03:52:19.735 INFO:teuthology.orchestra.run.smithi079.stdout:checking for uuid/uuid.h... yes 2023-06-08T03:52:19.763 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/uuid.h usability... no 2023-06-08T03:52:19.770 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/uuid.h presence... no 2023-06-08T03:52:19.770 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/uuid.h... no 2023-06-08T03:52:19.794 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/file.h usability... yes 2023-06-08T03:52:19.801 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/file.h presence... yes 2023-06-08T03:52:19.802 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/file.h... yes 2023-06-08T03:52:19.828 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/fcntl.h usability... yes 2023-06-08T03:52:19.835 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/fcntl.h presence... yes 2023-06-08T03:52:19.835 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/fcntl.h... yes 2023-06-08T03:52:19.860 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/syssgi.h usability... no 2023-06-08T03:52:19.869 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/syssgi.h presence... no 2023-06-08T03:52:19.870 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/syssgi.h... no 2023-06-08T03:52:19.908 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/param.h usability... yes 2023-06-08T03:52:19.918 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/param.h presence... yes 2023-06-08T03:52:19.918 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/param.h... yes 2023-06-08T03:52:19.922 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/stat.h... (cached) yes 2023-06-08T03:52:19.949 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/statvfs.h usability... yes 2023-06-08T03:52:19.956 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/statvfs.h presence... yes 2023-06-08T03:52:19.956 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/statvfs.h... yes 2023-06-08T03:52:19.981 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/time.h usability... yes 2023-06-08T03:52:19.988 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/time.h presence... yes 2023-06-08T03:52:19.989 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/time.h... yes 2023-06-08T03:52:20.027 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/ioctl.h usability... yes 2023-06-08T03:52:20.034 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/ioctl.h presence... yes 2023-06-08T03:52:20.034 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/ioctl.h... yes 2023-06-08T03:52:20.062 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/wait.h usability... yes 2023-06-08T03:52:20.070 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/wait.h presence... yes 2023-06-08T03:52:20.070 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/wait.h... yes 2023-06-08T03:52:20.075 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/types.h... (cached) yes 2023-06-08T03:52:20.080 INFO:teuthology.orchestra.run.smithi079.stdout:checking for strings.h... (cached) yes 2023-06-08T03:52:20.106 INFO:teuthology.orchestra.run.smithi079.stdout:checking err.h usability... yes 2023-06-08T03:52:20.113 INFO:teuthology.orchestra.run.smithi079.stdout:checking err.h presence... yes 2023-06-08T03:52:20.113 INFO:teuthology.orchestra.run.smithi079.stdout:checking for err.h... yes 2023-06-08T03:52:20.139 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/falloc.h usability... yes 2023-06-08T03:52:20.145 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/falloc.h presence... yes 2023-06-08T03:52:20.145 INFO:teuthology.orchestra.run.smithi079.stdout:checking for linux/falloc.h... yes 2023-06-08T03:52:20.170 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-06-08T03:52:20.179 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-06-08T03:52:20.180 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/fs/xfs_fsops.h... no 2023-06-08T03:52:20.202 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/fs/xfs_itable.h usability... no 2023-06-08T03:52:20.211 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/fs/xfs_itable.h presence... no 2023-06-08T03:52:20.211 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/fs/xfs_itable.h... no 2023-06-08T03:52:20.234 INFO:teuthology.orchestra.run.smithi079.stdout:checking xfs/platform_defs.h usability... no 2023-06-08T03:52:20.243 INFO:teuthology.orchestra.run.smithi079.stdout:checking xfs/platform_defs.h presence... no 2023-06-08T03:52:20.243 INFO:teuthology.orchestra.run.smithi079.stdout:checking for xfs/platform_defs.h... no 2023-06-08T03:52:20.265 INFO:teuthology.orchestra.run.smithi079.stdout:checking btrfs/ioctl.h usability... no 2023-06-08T03:52:20.275 INFO:teuthology.orchestra.run.smithi079.stdout:checking btrfs/ioctl.h presence... no 2023-06-08T03:52:20.275 INFO:teuthology.orchestra.run.smithi079.stdout:checking for btrfs/ioctl.h... no 2023-06-08T03:52:20.297 INFO:teuthology.orchestra.run.smithi079.stdout:checking cifs/ioctl.h usability... no 2023-06-08T03:52:20.306 INFO:teuthology.orchestra.run.smithi079.stdout:checking cifs/ioctl.h presence... no 2023-06-08T03:52:20.306 INFO:teuthology.orchestra.run.smithi079.stdout:checking for cifs/ioctl.h... no 2023-06-08T03:52:20.330 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/mman.h usability... yes 2023-06-08T03:52:20.338 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/mman.h presence... yes 2023-06-08T03:52:20.338 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/mman.h... yes 2023-06-08T03:52:20.352 INFO:teuthology.orchestra.run.smithi079.stdout:checking for xfs/xfs_log_format.h... no 2023-06-08T03:52:20.385 INFO:teuthology.orchestra.run.smithi079.stdout:checking for uuid_compare... no 2023-06-08T03:52:20.414 INFO:teuthology.orchestra.run.smithi079.stdout:checking for uuid_compare in -luuid... yes 2023-06-08T03:52:20.463 INFO:teuthology.orchestra.run.smithi079.stdout:checking for xfs/xfs.h... yes 2023-06-08T03:52:20.476 INFO:teuthology.orchestra.run.smithi079.stdout:checking for xfs/libxfs.h... no 2023-06-08T03:52:20.491 INFO:teuthology.orchestra.run.smithi079.stdout:checking whether xlog_assign_lsn is declared... no 2023-06-08T03:52:20.517 INFO:teuthology.orchestra.run.smithi079.stdout:checking for xfs/xqm.h... yes 2023-06-08T03:52:20.561 INFO:teuthology.orchestra.run.smithi079.stdout:checking xfsctl from xfs/xfs.h... ok 2023-06-08T03:52:20.583 INFO:teuthology.orchestra.run.smithi079.stdout:checking xfs/handle.h usability... yes 2023-06-08T03:52:20.589 INFO:teuthology.orchestra.run.smithi079.stdout:checking xfs/handle.h presence... yes 2023-06-08T03:52:20.589 INFO:teuthology.orchestra.run.smithi079.stdout:checking for xfs/handle.h... yes 2023-06-08T03:52:20.628 INFO:teuthology.orchestra.run.smithi079.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-06-08T03:52:20.658 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/acl.h usability... yes 2023-06-08T03:52:20.666 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/acl.h presence... yes 2023-06-08T03:52:20.666 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/acl.h... yes 2023-06-08T03:52:20.692 INFO:teuthology.orchestra.run.smithi079.stdout:checking acl/libacl.h usability... yes 2023-06-08T03:52:20.700 INFO:teuthology.orchestra.run.smithi079.stdout:checking acl/libacl.h presence... yes 2023-06-08T03:52:20.700 INFO:teuthology.orchestra.run.smithi079.stdout:checking for acl/libacl.h... yes 2023-06-08T03:52:20.731 INFO:teuthology.orchestra.run.smithi079.stdout:checking for acl_init in -lacl... yes 2023-06-08T03:52:20.752 INFO:teuthology.orchestra.run.smithi079.stdout:checking gdbm-ndbm.h usability... no 2023-06-08T03:52:20.762 INFO:teuthology.orchestra.run.smithi079.stdout:checking gdbm-ndbm.h presence... no 2023-06-08T03:52:20.762 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gdbm-ndbm.h... no 2023-06-08T03:52:20.782 INFO:teuthology.orchestra.run.smithi079.stdout:checking gdbm/ndbm.h usability... no 2023-06-08T03:52:20.791 INFO:teuthology.orchestra.run.smithi079.stdout:checking gdbm/ndbm.h presence... no 2023-06-08T03:52:20.791 INFO:teuthology.orchestra.run.smithi079.stdout:checking for gdbm/ndbm.h... no 2023-06-08T03:52:20.813 INFO:teuthology.orchestra.run.smithi079.stdout:checking ndbm.h usability... no 2023-06-08T03:52:20.834 INFO:teuthology.orchestra.run.smithi079.stdout:checking ndbm.h presence... no 2023-06-08T03:52:20.834 INFO:teuthology.orchestra.run.smithi079.stdout:checking for ndbm.h... no 2023-06-08T03:52:20.858 INFO:teuthology.orchestra.run.smithi079.stdout:checking libaio.h usability... yes 2023-06-08T03:52:20.868 INFO:teuthology.orchestra.run.smithi079.stdout:checking libaio.h presence... yes 2023-06-08T03:52:20.868 INFO:teuthology.orchestra.run.smithi079.stdout:checking for libaio.h... yes 2023-06-08T03:52:20.890 INFO:teuthology.orchestra.run.smithi079.stdout:checking liburing.h usability... no 2023-06-08T03:52:20.899 INFO:teuthology.orchestra.run.smithi079.stdout:checking liburing.h presence... no 2023-06-08T03:52:20.899 INFO:teuthology.orchestra.run.smithi079.stdout:checking for liburing.h... no 2023-06-08T03:52:20.920 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/capability.h usability... no 2023-06-08T03:52:20.929 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/capability.h presence... no 2023-06-08T03:52:20.929 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/capability.h... no 2023-06-08T03:52:20.951 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/fiemap.h usability... yes 2023-06-08T03:52:20.958 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/fiemap.h presence... yes 2023-06-08T03:52:20.958 INFO:teuthology.orchestra.run.smithi079.stdout:checking for linux/fiemap.h... yes 2023-06-08T03:52:20.990 INFO:teuthology.orchestra.run.smithi079.stdout:checking for fallocate... yes 2023-06-08T03:52:21.022 INFO:teuthology.orchestra.run.smithi079.stdout:checking for open_by_handle_at... yes 2023-06-08T03:52:21.045 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/prctl.h usability... yes 2023-06-08T03:52:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:checking sys/prctl.h presence... yes 2023-06-08T03:52:21.052 INFO:teuthology.orchestra.run.smithi079.stdout:checking for sys/prctl.h... yes 2023-06-08T03:52:21.075 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/fs.h usability... yes 2023-06-08T03:52:21.082 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/fs.h presence... yes 2023-06-08T03:52:21.082 INFO:teuthology.orchestra.run.smithi079.stdout:checking for linux/fs.h... yes 2023-06-08T03:52:21.102 INFO:teuthology.orchestra.run.smithi079.stdout:checking btrfsutil.h usability... no 2023-06-08T03:52:21.111 INFO:teuthology.orchestra.run.smithi079.stdout:checking btrfsutil.h presence... no 2023-06-08T03:52:21.111 INFO:teuthology.orchestra.run.smithi079.stdout:checking for btrfsutil.h... no 2023-06-08T03:52:21.152 INFO:teuthology.orchestra.run.smithi079.stdout:checking for copy_file_range... yes 2023-06-08T03:52:21.186 INFO:teuthology.orchestra.run.smithi079.stdout:checking for SEEK_DATA... yes 2023-06-08T03:52:21.230 INFO:teuthology.orchestra.run.smithi079.stdout:checking for BMV_OF_SHARED... yes 2023-06-08T03:52:21.267 INFO:teuthology.orchestra.run.smithi079.stdout:checking for nftw... yes 2023-06-08T03:52:21.334 INFO:teuthology.orchestra.run.smithi079.stdout:checking for RLIMIT_NOFILE... yes 2023-06-08T03:52:21.351 INFO:teuthology.orchestra.run.smithi079.stdout:checking for FIEXCHANGE_RANGE... no 2023-06-08T03:52:21.382 INFO:teuthology.orchestra.run.smithi079.stdout:checking for renameat2... yes 2023-06-08T03:52:21.413 INFO:teuthology.orchestra.run.smithi079.stdout:checking for reallocarray... yes 2023-06-08T03:52:21.426 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct mount_attr... no 2023-06-08T03:52:21.483 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_qgroup_limit... yes 2023-06-08T03:52:21.516 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-06-08T03:52:21.549 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-06-08T03:52:21.582 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-06-08T03:52:21.615 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-06-08T03:52:21.648 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-06-08T03:52:21.682 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-06-08T03:52:21.708 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/btrfs.h usability... yes 2023-06-08T03:52:21.715 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/btrfs.h presence... yes 2023-06-08T03:52:21.715 INFO:teuthology.orchestra.run.smithi079.stdout:checking for linux/btrfs.h... yes 2023-06-08T03:52:21.743 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/btrfs_tree.h usability... yes 2023-06-08T03:52:21.750 INFO:teuthology.orchestra.run.smithi079.stdout:checking linux/btrfs_tree.h presence... yes 2023-06-08T03:52:21.750 INFO:teuthology.orchestra.run.smithi079.stdout:checking for linux/btrfs_tree.h... yes 2023-06-08T03:52:21.780 INFO:teuthology.orchestra.run.smithi079.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-06-08T03:52:21.796 INFO:teuthology.orchestra.run.smithi079.stdout:configure: creating ./config.status 2023-06-08T03:52:22.082 INFO:teuthology.orchestra.run.smithi079.stdout:config.status: creating include/builddefs 2023-06-08T03:52:22.092 INFO:teuthology.orchestra.run.smithi079.stdout:config.status: creating include/config.h 2023-06-08T03:52:22.099 INFO:teuthology.orchestra.run.smithi079.stdout:config.status: executing libtool commands 2023-06-08T03:52:22.136 INFO:teuthology.orchestra.run.smithi079.stdout:Building include 2023-06-08T03:52:22.139 INFO:teuthology.orchestra.run.smithi079.stdout:Building lib 2023-06-08T03:52:22.142 INFO:teuthology.orchestra.run.smithi079.stdout: [LTDEP] 2023-06-08T03:52:22.250 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dataascii.lo 2023-06-08T03:52:22.361 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] databin.lo 2023-06-08T03:52:22.496 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] datapid.lo 2023-06-08T03:52:22.592 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] file_lock.lo 2023-06-08T03:52:22.728 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] forker.lo 2023-06-08T03:52:22.850 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] pattern.lo 2023-06-08T03:52:22.963 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] open_flags.lo 2023-06-08T03:52:23.094 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] random_range.lo 2023-06-08T03:52:23.300 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] string_to_tokens.lo 2023-06-08T03:52:23.397 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] str_to_bytes.lo 2023-06-08T03:52:23.517 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] tlibio.lo 2023-06-08T03:52:23.726 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] write_log.lo 2023-06-08T03:52:23.943 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] random.lo 2023-06-08T03:52:24.050 INFO:teuthology.orchestra.run.smithi079.stdout: [LD] 2023-06-08T03:52:24.120 INFO:teuthology.orchestra.run.smithi079.stdout:Building ltp 2023-06-08T03:52:24.122 INFO:teuthology.orchestra.run.smithi079.stdout: [DEP] 2023-06-08T03:52:24.244 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] doio 2023-06-08T03:52:24.827 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fsstress 2023-06-08T03:52:26.002 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fsx 2023-06-08T03:52:26.777 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] growfiles 2023-06-08T03:52:27.231 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] iogen 2023-06-08T03:52:27.547 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-stress 2023-06-08T03:52:27.921 INFO:teuthology.orchestra.run.smithi079.stdout:Building src 2023-06-08T03:52:27.926 INFO:teuthology.orchestra.run.smithi079.stdout: [DEP] 2023-06-08T03:52:29.204 INFO:teuthology.orchestra.run.smithi079.stdout: [DEP] 2023-06-08T03:52:29.229 INFO:teuthology.orchestra.run.smithi079.stdout: [DEP] 2023-06-08T03:52:29.236 INFO:teuthology.orchestra.run.smithi079.stdout: [DEP] 2023-06-08T03:52:29.413 INFO:teuthology.orchestra.run.smithi079.stdout: [DEP] 2023-06-08T03:52:29.434 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dirstress 2023-06-08T03:52:29.646 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fill 2023-06-08T03:52:29.787 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fill2 2023-06-08T03:52:29.990 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] getpagesize 2023-06-08T03:52:30.115 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] holes 2023-06-08T03:52:30.288 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] lstat64 2023-06-08T03:52:30.453 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] nametest 2023-06-08T03:52:30.665 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] permname 2023-06-08T03:52:30.845 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] randholes 2023-06-08T03:52:31.067 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] runas 2023-06-08T03:52:31.225 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] truncfile 2023-06-08T03:52:31.386 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] usemem 2023-06-08T03:52:31.522 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] mmapcat 2023-06-08T03:52:31.657 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] append_reader 2023-06-08T03:52:31.793 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] append_writer 2023-06-08T03:52:31.930 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dirperf 2023-06-08T03:52:32.126 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] metaperf 2023-06-08T03:52:32.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:52:32.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:52:32.355 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] devzero 2023-06-08T03:52:32.513 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] feature 2023-06-08T03:52:32.705 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] alloc 2023-06-08T03:52:32.894 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fault 2023-06-08T03:52:33.042 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fstest 2023-06-08T03:52:33.253 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_access_root 2023-06-08T03:52:33.393 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] godown 2023-06-08T03:52:33.544 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] resvtest 2023-06-08T03:52:33.710 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] writemod 2023-06-08T03:52:33.842 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] writev_on_pagefault 2023-06-08T03:52:33.985 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] makeextents 2023-06-08T03:52:34.144 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] itrash 2023-06-08T03:52:34.286 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] rename 2023-06-08T03:52:34.414 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] multi_open_unlink 2023-06-08T03:52:34.563 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] unwritten_sync 2023-06-08T03:52:34.725 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] genhashnames 2023-06-08T03:52:34.894 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_holes 2023-06-08T03:52:35.028 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_writev 2023-06-08T03:52:35.166 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_truncate_cmtime 2023-06-08T03:52:35.310 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dirhash_collide 2023-06-08T03:52:35.473 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_rename_overwrite 2023-06-08T03:52:35.602 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] holetest 2023-06-08T03:52:35.797 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_truncate_self 2023-06-08T03:52:35.926 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] af_unix 2023-06-08T03:52:36.065 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_stale_pmd 2023-06-08T03:52:36.203 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_cow_race 2023-06-08T03:52:36.352 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_fallocate 2023-06-08T03:52:36.489 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fsync-err 2023-06-08T03:52:36.655 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_write_ro 2023-06-08T03:52:36.792 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_ext4_dax_journal_corruption 2023-06-08T03:52:36.936 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_ext4_dax_inline_corruption 2023-06-08T03:52:37.077 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_ofd_locks 2023-06-08T03:52:37.258 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_collision 2023-06-08T03:52:37.436 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] mmap-write-concurrent 2023-06-08T03:52:37.590 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_get_file_time 2023-06-08T03:52:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_create_short_dirs 2023-06-08T03:52:37.876 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_create_long_dirs 2023-06-08T03:52:38.026 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_enospc 2023-06-08T03:52:38.194 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_writev_overlap 2023-06-08T03:52:38.348 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] checkpoint_journal 2023-06-08T03:52:38.487 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] mmap-rw-fault 2023-06-08T03:52:38.650 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] allocstale 2023-06-08T03:52:38.802 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_cow_memory_failure 2023-06-08T03:52:38.956 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fake-dump-rootino 2023-06-08T03:52:39.092 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dio-buf-fault 2023-06-08T03:52:39.232 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] xfsctl 2023-06-08T03:52:39.400 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] bstat 2023-06-08T03:52:39.583 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mtab 2023-06-08T03:52:39.749 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] getdevicesize 2023-06-08T03:52:39.878 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] preallo_rw_pattern_reader 2023-06-08T03:52:40.021 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] preallo_rw_pattern_writer 2023-06-08T03:52:40.165 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] ftrunc 2023-06-08T03:52:40.317 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] trunc 2023-06-08T03:52:40.481 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fs_perms 2023-06-08T03:52:40.630 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] testx 2023-06-08T03:52:40.752 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] looptest 2023-06-08T03:52:40.926 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] locktest 2023-06-08T03:52:41.200 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] unwritten_mmap 2023-06-08T03:52:41.347 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] bulkstat_unlink_test 2023-06-08T03:52:41.527 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] deduperace 2023-06-08T03:52:41.703 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] bulkstat_unlink_test_modified 2023-06-08T03:52:41.879 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_dir_offset 2023-06-08T03:52:42.014 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_futimens 2023-06-08T03:52:42.143 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_immutable 2023-06-08T03:52:42.681 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] stale_handle 2023-06-08T03:52:42.834 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] pwrite_mmap_blocked 2023-06-08T03:52:42.973 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_dir_offset2 2023-06-08T03:52:43.147 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] seek_sanity_test 2023-06-08T03:52:43.517 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] seek_copy_test 2023-06-08T03:52:43.677 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_readdir_1 2023-06-08T03:52:43.812 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_readdir_2 2023-06-08T03:52:43.950 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fsync-tester 2023-06-08T03:52:44.172 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] nsexec 2023-06-08T03:52:44.332 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] cloner 2023-06-08T03:52:44.491 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] renameat2 2023-06-08T03:52:44.641 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_getcwd 2023-06-08T03:52:44.785 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] e4compact 2023-06-08T03:52:44.966 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] test-nextquota 2023-06-08T03:52:45.120 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] punch-alternating 2023-06-08T03:52:45.275 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] attr-list-by-handle-cursor-test 2023-06-08T03:52:45.426 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] listxattr 2023-06-08T03:52:45.560 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dio-interleaved 2023-06-08T03:52:45.706 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_dir_type 2023-06-08T03:52:45.959 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] dio-invalidate-cache 2023-06-08T03:52:46.186 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] stat_test 2023-06-08T03:52:46.452 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_encrypted_d_revalidate 2023-06-08T03:52:46.602 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] attr_replace_test 2023-06-08T03:52:46.750 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] swapon 2023-06-08T03:52:46.899 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] mkswap 2023-06-08T03:52:47.039 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_attr_corruption 2023-06-08T03:52:47.175 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_open_tmpfiles 2023-06-08T03:52:47.336 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fscrypt-crypt-util 2023-06-08T03:52:47.903 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] bulkstat_null_ocount 2023-06-08T03:52:48.047 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] splice-test 2023-06-08T03:52:48.211 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] chprojid_fail 2023-06-08T03:52:48.358 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] detached_mounts_propagation 2023-06-08T03:52:48.523 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] ext4_resize 2023-06-08T03:52:48.658 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_readdir_3 2023-06-08T03:52:48.834 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] splice2pipe 2023-06-08T03:52:48.983 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] uuid_ioctl 2023-06-08T03:52:49.124 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fiemap-tester 2023-06-08T03:52:49.363 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_stripealign 2023-06-08T03:52:49.503 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] open_by_handle 2023-06-08T03:52:49.732 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] t_mmap_dio 2023-06-08T03:52:49.880 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] xfsfind 2023-06-08T03:52:50.054 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] fssum 2023-06-08T03:52:50.440 INFO:teuthology.orchestra.run.smithi079.stdout:Building log-writes 2023-06-08T03:52:50.445 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] replay-log 2023-06-08T03:52:50.722 INFO:teuthology.orchestra.run.smithi079.stdout:Building perf 2023-06-08T03:52:50.725 INFO:teuthology.orchestra.run.smithi079.stdout:Building aio-dio-regress 2023-06-08T03:52:50.731 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-append-write-fallocate-race 2023-06-08T03:52:50.889 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-fcntl-race 2023-06-08T03:52:51.040 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-subblock-eof-read 2023-06-08T03:52:51.174 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-append-write-read-race 2023-06-08T03:52:51.336 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-invalidate-readahead 2023-06-08T03:52:51.487 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-write-verify 2023-06-08T03:52:51.663 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-06-08T03:52:51.790 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-last-ref-held-by-io 2023-06-08T03:52:51.950 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-cycle-write 2023-06-08T03:52:52.121 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-extend-stat 2023-06-08T03:52:52.274 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-06-08T03:52:52.398 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-hole-filling-race 2023-06-08T03:52:52.537 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-invalidate-failure 2023-06-08T03:52:52.685 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aiodio_sparse2 2023-06-08T03:52:52.881 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aiocp 2023-06-08T03:52:53.085 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-cow-race 2023-06-08T03:52:53.228 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] aio-dio-eof-race 2023-06-08T03:52:53.398 INFO:teuthology.orchestra.run.smithi079.stdout:Building vfs 2023-06-08T03:52:53.403 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] vfstest 2023-06-08T03:52:55.061 INFO:teuthology.orchestra.run.smithi079.stdout: [CC] mount-idmapped 2023-06-08T03:52:55.390 INFO:teuthology.orchestra.run.smithi079.stdout:Building m4 2023-06-08T03:52:55.393 INFO:teuthology.orchestra.run.smithi079.stdout:Building common 2023-06-08T03:52:55.396 INFO:teuthology.orchestra.run.smithi079.stdout:Building tests 2023-06-08T03:52:55.401 INFO:teuthology.orchestra.run.smithi079.stdout:Building perf 2023-06-08T03:52:55.406 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/perf/group.list 2023-06-08T03:52:55.416 INFO:teuthology.orchestra.run.smithi079.stdout:Building ext4 2023-06-08T03:52:55.421 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ext4/group.list 2023-06-08T03:52:55.454 INFO:teuthology.orchestra.run.smithi079.stdout:Building ocfs2 2023-06-08T03:52:55.459 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ocfs2/group.list 2023-06-08T03:52:55.470 INFO:teuthology.orchestra.run.smithi079.stdout:Building overlay 2023-06-08T03:52:55.475 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/overlay/group.list 2023-06-08T03:52:55.509 INFO:teuthology.orchestra.run.smithi079.stdout:Building cifs 2023-06-08T03:52:55.514 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/cifs/group.list 2023-06-08T03:52:55.525 INFO:teuthology.orchestra.run.smithi079.stdout:Building generic 2023-06-08T03:52:55.531 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/generic/group.list 2023-06-08T03:52:55.618 INFO:teuthology.orchestra.run.smithi079.stdout:Building shared 2023-06-08T03:52:55.623 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/shared/group.list 2023-06-08T03:52:55.638 INFO:teuthology.orchestra.run.smithi079.stdout:Building nfs 2023-06-08T03:52:55.643 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/nfs/group.list 2023-06-08T03:52:55.655 INFO:teuthology.orchestra.run.smithi079.stdout:Building f2fs 2023-06-08T03:52:55.660 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/f2fs/group.list 2023-06-08T03:52:55.675 INFO:teuthology.orchestra.run.smithi079.stdout:Building udf 2023-06-08T03:52:55.680 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/udf/group.list 2023-06-08T03:52:55.690 INFO:teuthology.orchestra.run.smithi079.stdout:Building ceph 2023-06-08T03:52:55.694 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ceph/group.list 2023-06-08T03:52:55.707 INFO:teuthology.orchestra.run.smithi079.stdout:Building xfs 2023-06-08T03:52:55.713 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/xfs/group.list 2023-06-08T03:52:55.802 INFO:teuthology.orchestra.run.smithi079.stdout:Building btrfs 2023-06-08T03:52:55.807 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/btrfs/group.list 2023-06-08T03:52:55.859 INFO:teuthology.orchestra.run.smithi079.stdout:Building tmpfs 2023-06-08T03:52:55.864 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/tmpfs/group.list 2023-06-08T03:52:55.875 INFO:teuthology.orchestra.run.smithi079.stdout:Building selftest 2023-06-08T03:52:55.880 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/selftest/group.list 2023-06-08T03:52:55.893 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo make install) 2023-06-08T03:52:55.929 INFO:teuthology.orchestra.run.smithi079.stdout:Building include 2023-06-08T03:52:55.932 INFO:teuthology.orchestra.run.smithi079.stdout:Building lib 2023-06-08T03:52:55.936 INFO:teuthology.orchestra.run.smithi079.stdout:Building ltp 2023-06-08T03:52:55.940 INFO:teuthology.orchestra.run.smithi079.stdout:Building src 2023-06-08T03:52:55.972 INFO:teuthology.orchestra.run.smithi079.stdout:Building log-writes 2023-06-08T03:52:55.975 INFO:teuthology.orchestra.run.smithi079.stdout:Building perf 2023-06-08T03:52:55.979 INFO:teuthology.orchestra.run.smithi079.stdout:Building aio-dio-regress 2023-06-08T03:52:55.982 INFO:teuthology.orchestra.run.smithi079.stdout:Building vfs 2023-06-08T03:52:55.986 INFO:teuthology.orchestra.run.smithi079.stdout:Building m4 2023-06-08T03:52:55.989 INFO:teuthology.orchestra.run.smithi079.stdout:Building common 2023-06-08T03:52:55.992 INFO:teuthology.orchestra.run.smithi079.stdout:Building tests 2023-06-08T03:52:55.997 INFO:teuthology.orchestra.run.smithi079.stdout:Building perf 2023-06-08T03:52:56.002 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/perf/group.list 2023-06-08T03:52:56.012 INFO:teuthology.orchestra.run.smithi079.stdout:Building ext4 2023-06-08T03:52:56.017 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ext4/group.list 2023-06-08T03:52:56.051 INFO:teuthology.orchestra.run.smithi079.stdout:Building ocfs2 2023-06-08T03:52:56.055 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ocfs2/group.list 2023-06-08T03:52:56.066 INFO:teuthology.orchestra.run.smithi079.stdout:Building overlay 2023-06-08T03:52:56.071 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/overlay/group.list 2023-06-08T03:52:56.105 INFO:teuthology.orchestra.run.smithi079.stdout:Building cifs 2023-06-08T03:52:56.110 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/cifs/group.list 2023-06-08T03:52:56.120 INFO:teuthology.orchestra.run.smithi079.stdout:Building generic 2023-06-08T03:52:56.127 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/generic/group.list 2023-06-08T03:52:56.212 INFO:teuthology.orchestra.run.smithi079.stdout:Building shared 2023-06-08T03:52:56.217 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/shared/group.list 2023-06-08T03:52:56.232 INFO:teuthology.orchestra.run.smithi079.stdout:Building nfs 2023-06-08T03:52:56.237 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/nfs/group.list 2023-06-08T03:52:56.249 INFO:teuthology.orchestra.run.smithi079.stdout:Building f2fs 2023-06-08T03:52:56.254 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/f2fs/group.list 2023-06-08T03:52:56.269 INFO:teuthology.orchestra.run.smithi079.stdout:Building udf 2023-06-08T03:52:56.274 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/udf/group.list 2023-06-08T03:52:56.284 INFO:teuthology.orchestra.run.smithi079.stdout:Building ceph 2023-06-08T03:52:56.289 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ceph/group.list 2023-06-08T03:52:56.301 INFO:teuthology.orchestra.run.smithi079.stdout:Building xfs 2023-06-08T03:52:56.308 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/xfs/group.list 2023-06-08T03:52:56.394 INFO:teuthology.orchestra.run.smithi079.stdout:Building btrfs 2023-06-08T03:52:56.400 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/btrfs/group.list 2023-06-08T03:52:56.451 INFO:teuthology.orchestra.run.smithi079.stdout:Building tmpfs 2023-06-08T03:52:56.455 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/tmpfs/group.list 2023-06-08T03:52:56.467 INFO:teuthology.orchestra.run.smithi079.stdout:Building selftest 2023-06-08T03:52:56.472 INFO:teuthology.orchestra.run.smithi079.stdout: [GROUP] /tmp/tmp.l6an1raz8Jxfstests-dev/tests/selftest/group.list 2023-06-08T03:52:56.482 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-06-08T03:52:56.484 INFO:teuthology.orchestra.run.smithi079.stdout:gmake[1]: Nothing to be done for 'install'. 2023-06-08T03:52:56.484 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-06-08T03:52:56.486 INFO:teuthology.orchestra.run.smithi079.stdout:gmake[1]: Nothing to be done for 'install'. 2023-06-08T03:52:56.486 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-06-08T03:52:56.488 INFO:teuthology.orchestra.run.smithi079.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-06-08T03:52:56.497 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-06-08T03:52:56.572 INFO:teuthology.orchestra.run.smithi079.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-06-08T03:52:56.581 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-06-08T03:52:56.597 INFO:teuthology.orchestra.run.smithi079.stdout:Building log-writes 2023-06-08T03:52:56.600 INFO:teuthology.orchestra.run.smithi079.stdout:Building perf 2023-06-08T03:52:56.603 INFO:teuthology.orchestra.run.smithi079.stdout:Building aio-dio-regress 2023-06-08T03:52:56.607 INFO:teuthology.orchestra.run.smithi079.stdout:Building vfs 2023-06-08T03:52:56.610 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake -C log-writes install 2023-06-08T03:52:56.611 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-06-08T03:52:56.620 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-06-08T03:52:56.629 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake -C perf install 2023-06-08T03:52:56.630 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-06-08T03:52:56.638 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-06-08T03:52:56.660 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-06-08T03:52:56.662 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-06-08T03:52:56.671 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 aio-dio-append-write-fallocate-race aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-last-ref-held-by-io aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aio-dio-hole-filling-race aio-dio-invalidate-failure aiodio_sparse2 aiocp aio-dio-cow-race aio-dio-eof-race /var/lib/xfstests/src/aio-dio-regress 2023-06-08T03:52:56.722 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake -C vfs install 2023-06-08T03:52:56.723 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-06-08T03:52:56.732 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-06-08T03:52:56.743 INFO:teuthology.orchestra.run.smithi079.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-06-08T03:52:56.752 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-06-08T03:52:57.931 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py /var/lib/xfstests/src 2023-06-08T03:52:58.024 INFO:teuthology.orchestra.run.smithi079.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-06-08T03:52:58.054 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-06-08T03:52:58.055 INFO:teuthology.orchestra.run.smithi079.stdout:gmake[1]: Nothing to be done for 'install'. 2023-06-08T03:52:58.056 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-06-08T03:52:58.057 INFO:teuthology.orchestra.run.smithi079.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-06-08T03:52:58.065 INFO:teuthology.orchestra.run.smithi079.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-06-08T03:52:58.193 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-06-08T03:52:58.195 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/btrfs/ install 2023-06-08T03:52:58.198 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-06-08T03:52:58.206 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 290 291 299 /var/lib/xfstests/tests/btrfs 2023-06-08T03:52:58.984 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-06-08T03:52:58.992 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 057.out 175.out 186.out 240.out 069.out 092.out 077.out 140.out 265.out 236.out 108.out 204.out 177.out 194.out 234.out 249.out 109.out 033.out 082.out 252.out 270.out 040.out 176.out 202.out 137.out 034.out 125.out 014.out 165.out 181.out 152.out 142.out 264.out 192.out 148.out 282.out 267.out 052.out 228.out 183.out 094.out 273.out 251.out 110.out 043.out 209.out 151.out 206.out 195.out 274.out 215.out 115.out 015.out 250.out 191.out 281.out 046.out 051.out 244.out 219.out 119.out 019.out 128.out 232.out 248.out 162.out 147.out 197.out 196.out 242.out 004.out 189.out 220.out 120.out 020.out 102.out 263.out 171.out 261.out 045.out 217.out 117.out 017.out 072.out 068.out 048.out 062.out 038.out 084.out 129.out 233.out 066.out 029.out 229.out 103.out 254.out 111.out 011.out 243.out 275.out 134.out 170.out 260.out 157.out 143.out 133.out 286.out 161.out 246.out 216.out 116.out 016.out 155.out 118.out 287.out 067.out 010.out 238.out 138.out 095.out 184.out 160.out 190.out 291.out 081.out 024.out 071.out 061.out 047.out 200.out 100.out 025.out 139.out 258.out 185.out 269.out 208.out 039.out 222.out 035.out 169.out 098.out 290.out 080.out 070.out 060.out 207.out 106.out 009.out 167.out 003.out 211.out 158.out 085.out 146.out 188.out 104.out 089.out 086.out 279.out 225.out 174.out 156.out 135.out 257.out 124.out 172.out 058.out 262.out 036.out 042.out 193.out 097.out 271.out 283.out 083.out 199.out 099.out 278.out 053.out 164.out 239.out 241.out 002.out 178.out 141.out 214.out 114.out 074.out 056.out 012.out 041.out 223.out 150.out 123.out 179.out 144.out 032.out 277.out 205.out 136.out 005.out 096.out 198.out 008.out 280.out 050.out 105.out 027.out 187.out 091.out 153.out 088.out 285.out 044.out 201.out 101.out 001.out 266.out 237.out 065.out 231.out 227.out 075.out 131.out 213.out 113.out 013.out 031.out 182.out 272.out 078.out 173.out 259.out 210.out 132.out 037.out 093.out 235.out 127.out 230.out 166.out 130.out 023.out 073.out 030.out 203.out 063.out 055.out 245.out 145.out 159.out 122.out 076.out 022.out 006.out 284.out 299.out 224.out 154.out 276.out 149.out 064.out 218.out 018.out 168.out 007.out 079.out 054.out 059.out 049.out 226.out 163.out 126.out 121.out 026.out 180.out 268.out 253.out 255.out 090.out 221.out 087.out 212.out 112.out 256.out 021.out 247.out 028.out 107.out /var/lib/xfstests/tests/btrfs 2023-06-08T03:52:59.773 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ceph/ install 2023-06-08T03:52:59.774 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-06-08T03:52:59.783 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-06-08T03:52:59.802 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-06-08T03:52:59.811 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-06-08T03:52:59.830 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/cifs/ install 2023-06-08T03:52:59.831 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-06-08T03:52:59.840 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-06-08T03:52:59.848 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-06-08T03:52:59.857 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-06-08T03:52:59.865 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ext4/ install 2023-06-08T03:52:59.867 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-06-08T03:52:59.875 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-06-08T03:53:00.061 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-06-08T03:53:00.069 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 032.out 039.out 305.out 005.out 008.out 035.out 036.out 308.out 050.out 040.out 306.out 007.out 021.out 046.out 012.out 009.out 033.out 045.out 034.out 056.out 047.out 018.out 023.out 302.out 020.out 002.out 014.out 024.out 013.out 031.out 015.out 304.out 004.out 048.out 038.out 052.out 303.out 029.out 011.out 027.out 042.out 044.out 055.out 030.out 058.out 003.out 017.out 043.out 053.out 016.out 059.out 037.out 301.out 010.out 022.out 006.out 057.out 025.out 026.out 271.out 051.out 041.out 307.out 019.out 028.out 054.out 049.out /var/lib/xfstests/tests/ext4 2023-06-08T03:53:00.265 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/f2fs/ install 2023-06-08T03:53:00.267 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-06-08T03:53:00.275 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-06-08T03:53:00.286 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-06-08T03:53:00.295 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-06-08T03:53:00.306 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/generic/ install 2023-06-08T03:53:00.311 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-06-08T03:53:00.319 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 /var/lib/xfstests/tests/generic 2023-06-08T03:53:02.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:53:02.248 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:53:02.274 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-06-08T03:53:02.282 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 050.out.xfsquota 035.out.nfs 035.out.default 050.out.nojournal 050.out.default 050.cfg 035.cfg 208.out 433.out 670.out 040.out 104.out 575.out 537.out 683.out 497.out 337.out 702.out 460.out 149.out 088.out 366.out 202.out 137.out 034.out 110.out 389.out 403.out 593.out 571.out 243.out 592.out 282.out 472.out 468.out 628.out 528.out 607.out 428.out 659.out 407.out 228.out 128.out 171.out 028.out 620.out 273.out 555.out 492.out 609.out 707.out 693.out 595.out 506.out 268.out 632.out 532.out 591.out 281.out 471.out 619.out 255.out 013.out 419.out 319.out 177.out 523.out 119.out 019.out 727.out 372.out 645.out 673.out 190.out 686.out 312.out 718.out 143.out 716.out 161.out 516.out 518.out 538.out 655.out 216.out 116.out 016.out 155.out 338.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 457.out 333.out 258.out 362.out 379.out 691.out 039.out 173.out 498.out 665.out 098.out 711.out 290.out 480.out 706.out 060.out 406.out 306.out 106.out 642.out 447.out 217.out 120.out 191.out 411.out 481.out 211.out 158.out 014.out 182.out 370.out 188.out 030.out 099.out 585.out 724.out 624.out 524.out 174.out 156.out 324.out 224.out 313.out 355.out 124.out 672.out 278.out 063.out 058.out 384.out 296.out 042.out 193.out 271.out 465.out 473.out 335.out 247.out 509.out 568.out 309.out 441.out 440.out 475.out 548.out 356.out 341.out 094.out 241.out 600.out 089.out 714.out 614.out 514.out 414.out 314.out 416.out 214.out 381.out 342.out 041.out 664.out 340.out 358.out 572.out 586.out 521.out 240.out 549.out 095.out 633.out 434.out 140.out 708.out 533.out 508.out 408.out 599.out 476.out 087.out 199.out 458.out 233.out 639.out 721.out 540.out 397.out 053.out 279.out 276.out 448.out 164.out 376.out 589.out 587.out 339.out 294.out 352.out 484.out 489.out 139.out 317.out 399.out 220.out 552.out 695.out 017.out 677.out 175.out 315.out 295.out 580.out 075.out 186.out 452.out 679.out 292.out 252.out 077.out 167.out 688.out 206.out 348.out 394.out 651.out 184.out 466.out 697.out 495.out 725.out 109.out 482.out 625.out 283.out 525.out 488.out 496.out 325.out 551.out 225.out 322.out 636.out 125.out 715.out 192.out 544.out 515.out 650.out 152.out 658.out 563.out 393.out 578.out 451.out 015.out 392.out 682.out 684.out 550.out 052.out 652.out 534.out 351.out 583.out 450.out 463.out 251.out 267.out 176.out 069.out 644.out 065.out 487.out 615.out 395.out 676.out 415.out 344.out 215.out 115.out 444.out 027.out 250.out 391.out 674.out 464.out 705.out 536.out 577.out 439.out 244.out 336.out 144.out 505.out 405.out 057.out 205.out 136.out 598.out 363.out 036.out 198.out 723.out 008.out 390.out 680.out 605.out 305.out 105.out 005.out 367.out 291.out 719.out 323.out 685.out 562.out 382.out 570.out 388.out 527.out 237.out 151.out 227.out 163.out 044.out 332.out 661.out 582.out 232.out 478.out 584.out 462.out 597.out 412.out 132.out 179.out 037.out 293.out 499.out 638.out 353.out 020.out 660.out 561.out 141.out 076.out 668.out 648.out 387.out 594.out 438.out 265.out 289.out 574.out 006.out 690.out 284.out 519.out 567.out 299.out 474.out 456.out 461.out 477.out 307.out 647.out 493.out 107.out 007.out 380.out 526.out 479.out 426.out 150.out 326.out 082.out 226.out 413.out 126.out 025.out 121.out 026.out 021.out 485.out 694.out 347.out 275.out 090.out 361.out 564.out 302.out 432.out 002.out 558.out 162.out 147.out 288.out 197.out 033.out 396.out 051.out 350.out 547.out 047.out 024.out 360.out 543.out 261.out 045.out 692.out 443.out 726.out 378.out 346.out 048.out 062.out 626.out 529.out 231.out 343.out 662.out 425.out 329.out 165.out 334.out 103.out 003.out 129.out 423.out 371.out 234.out 183.out 671.out 486.out 223.out 123.out 262.out 669.out 023.out 459.out 616.out 032.out 266.out 146.out 398.out 696.out 427.out 590.out 280.out 470.out 455.out 187.out 091.out 046.out 374.out 631.out 328.out 260.out 654.out 359.out 531.out 649.out 301.out 101.out 635.out 623.out 385.out 331.out 556.out 713.out 613.out 513.out 622.out 131.out 213.out 634.out 031.out 565.out 630.out 675.out 272.out 530.out 510.out 627.out 573.out 576.out 430.out 310.out 569.out 454.out 259.out 210.out 449.out 327.out 330.out 067.out 093.out 604.out 287.out 127.out 699.out 166.out 130.out 219.out 073.out 603.out 503.out 303.out 203.out 354.out 157.out 386.out 349.out 055.out 245.out 709.out 522.out 422.out 222.out 159.out 539.out 249.out 122.out 689.out 022.out 681.out 424.out 321.out 066.out 490.out 435.out 375.out 134.out 154.out 566.out 618.out 501.out 418.out 318.out 542.out 218.out 118.out 018.out 377.out 172.out 168.out 079.out 469.out 054.out 059.out 049.out 442.out 657.out 621.out 180.out 270.out 253.out 421.out 712.out 612.out 667.out 512.out 494.out 221.out 212.out 112.out 256.out 012.out 029.out 195.out 554.out 148.out 185.out 596.out 720.out 248.out 196.out 520.out 242.out 420.out 285.out 320.out 629.out 230.out 602.out 502.out 402.out 102.out 717.out 617.out 517.out 429.out 417.out 365.out 092.out 117.out 357.out 072.out 068.out 229.out 038.out 084.out 056.out 369.out 611.out 511.out 311.out 368.out 637.out 111.out 011.out 257.out 194.out 170.out 606.out 133.out 286.out 043.out 437.out 545.out 559.out 710.out 610.out 687.out 410.out 701.out 601.out 401.out 201.out 010.out 001.out 096.out 114.out 113.out 074.out 581.out 145.out 641.out 491.out 081.out 071.out 083.out 653.out 700.out 064.out 400.out 300.out 200.out 100.out 646.out 541.out 445.out 269.out 546.out 656.out 698.out 553.out 178.out 298.out 446.out 080.out 070.out 608.out 507.out 409.out 308.out 209.out 108.out 009.out 643.out 004.out 557.out 453.out 666.out 236.out 078.out 364.out 097.out 383.out 085.out 704.out 588.out 504.out 404.out 304.out 204.out 535.out 345.out 246.out 467.out 086.out 254.out 207.out 142.out 235.out 436.out 703.out 135.out 264.out 431.out 678.out 373.out 316.out 169.out 640.out 579.out 297.out 153.out 560.out 663.out /var/lib/xfstests/tests/generic 2023-06-08T03:53:04.264 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/nfs/ install 2023-06-08T03:53:04.265 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-06-08T03:53:04.273 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-06-08T03:53:04.282 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-06-08T03:53:04.290 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-06-08T03:53:04.299 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/ocfs2/ install 2023-06-08T03:53:04.300 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-06-08T03:53:04.309 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-06-08T03:53:04.317 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-06-08T03:53:04.326 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-06-08T03:53:04.335 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/overlay/ install 2023-06-08T03:53:04.336 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-06-08T03:53:04.345 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-06-08T03:53:04.603 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-06-08T03:53:04.612 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 019.out 034.out 018.out 024.out 077.out 069.out 020.out 102.out 014.out 013.out 031.out 015.out 117.out 017.out 072.out 068.out 063.out 048.out 062.out 038.out 026.out 042.out 052.out 110.out 067.out 029.out 111.out 055.out 112.out 073.out 078.out 030.out 058.out 003.out 045.out 023.out 043.out 053.out 109.out 036.out 116.out 016.out 065.out 010.out 001.out 022.out 106.out 007.out 006.out 115.out 057.out 113.out 114.out 074.out 046.out 071.out 061.out 025.out 051.out 011.out 041.out 047.out 100.out 064.out 107.out 104.out 028.out 076.out 054.out 059.out 008.out 049.out 032.out 039.out 021.out 005.out 002.out 035.out 108.out 070.out 060.out 050.out 027.out 105.out 009.out 004.out 037.out 044.out 012.out 101.out 103.out 033.out 066.out 040.out 056.out 075.out /var/lib/xfstests/tests/overlay 2023-06-08T03:53:04.875 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/perf/ install 2023-06-08T03:53:04.877 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-06-08T03:53:04.885 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-06-08T03:53:04.894 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-06-08T03:53:04.903 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-06-08T03:53:04.911 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/selftest/ install 2023-06-08T03:53:04.913 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-06-08T03:53:04.921 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-06-08T03:53:04.943 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-06-08T03:53:04.951 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 006.out 003.out 005.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-06-08T03:53:04.973 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/shared/ install 2023-06-08T03:53:04.975 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-06-08T03:53:04.983 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-06-08T03:53:04.997 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-06-08T03:53:05.005 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-06-08T03:53:05.019 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/tmpfs/ install 2023-06-08T03:53:05.020 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-06-08T03:53:05.029 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-06-08T03:53:05.038 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-06-08T03:53:05.046 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-06-08T03:53:05.055 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/udf/ install 2023-06-08T03:53:05.056 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-06-08T03:53:05.065 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-06-08T03:53:05.073 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-06-08T03:53:05.082 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-06-08T03:53:05.090 INFO:teuthology.orchestra.run.smithi079.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.l6an1raz8Jxfstests-dev/tests/xfs/ install 2023-06-08T03:53:05.094 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-06-08T03:53:05.103 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-06-08T03:53:06.944 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-06-08T03:53:06.953 INFO:teuthology.orchestra.run.smithi079.stdout:../../install-sh -o root -g root -m 644 096.out.internal 033.out.crc 096.out.external 033.out.default 071.out.64 216.out.classic 216.out.64mblog 071.out.32 216.cfg 033.cfg 040.out 304.out 349.out 457.out 537.out 295.out 497.out 337.out 385.out 082.out 270.out 460.out 149.out 759.out 502.out 088.out 366.out 202.out 137.out 746.out 110.out 468.out 018.out 593.out 571.out 567.out 592.out 282.out 267.out 728.out 178.out 799.out 528.out 769.out 428.out 507.out 407.out 228.out 128.out 171.out 183.out 247.out 273.out 474.out 109.out 595.out 506.out 732.out 268.out 035.out 113.out 532.out 591.out 281.out 471.out 723.out 519.out 319.out 467.out 523.out 119.out 775.out 727.out 372.out 503.out 526.out 329.out 165.out 229.out 103.out 129.out 371.out 234.out 738.out 078.out 718.out 143.out 716.out 161.out 516.out 416.out 116.out 043.out 155.out 338.out 067.out 238.out 500.out 138.out 189.out 263.out 277.out 181.out 160.out 483.out 274.out 722.out 061.out 239.out 047.out 772.out 258.out 379.out 322.out 039.out 173.out 222.out 742.out 098.out 426.out 290.out 480.out 060.out 406.out 306.out 141.out 106.out 006.out 167.out 191.out 017.out 542.out 211.out 783.out 485.out 442.out 099.out 177.out 174.out 156.out 771.out 324.out 224.out 313.out 124.out 758.out 278.out 058.out 741.out 384.out 773.out 042.out 193.out 392.out 768.out 399.out 473.out 770.out 709.out 755.out 364.out 209.out 441.out 440.out 157.out 740.out 356.out 197.out 341.out 187.out 094.out 241.out 501.out 089.out 540.out 365.out 514.out 414.out 314.out 214.out 381.out 056.out 041.out 340.out 793.out 358.out 572.out 568.out 586.out 240.out 549.out 092.out 439.out 434.out 140.out 708.out 265.out 508.out 408.out 308.out 208.out 737.out 204.out 535.out 104.out 733.out 544.out 059.out 249.out 335.out 142.out 235.out 575.out 253.out 715.out 533.out 264.out 431.out 784.out 433.out 316.out 169.out 579.out 297.out 393.out 333.out 476.out 087.out 199.out 739.out 068.out 233.out 752.out 736.out 443.out 086.out 053.out 749.out 279.out 276.out 711.out 164.out 589.out 587.out 339.out 577.out 294.out 352.out 458.out 498.out 489.out 139.out 774.out 790.out 552.out 781.out 357.out 065.out 186.out 452.out 069.out 252.out 585.out 751.out 724.out 206.out 190.out 794.out 348.out 184.out 435.out 466.out 248.out 725.out 785.out 482.out 750.out 525.out 496.out 325.out 176.out 225.out 125.out 789.out 192.out 296.out 152.out 563.out 246.out 551.out 436.out 792.out 075.out 472.out 578.out 463.out 550.out 052.out 534.out 351.out 430.out 583.out 305.out 451.out 425.out 450.out 744.out 251.out 444.out 487.out 762.out 465.out 395.out 515.out 415.out 389.out 315.out 215.out 115.out 015.out 027.out 250.out 344.out 046.out 051.out 536.out 254.out 030.out 244.out 336.out 028.out 150.out 144.out 505.out 342.out 136.out 005.out 036.out 198.out 008.out 390.out 405.out 588.out 205.out 105.out 367.out 291.out 437.out 719.out 323.out 363.out 562.out 382.out 570.out 388.out 527.out 158.out 766.out 019.out 237.out 151.out 760.out 227.out 163.out 332.out 796.out 232.out 478.out 584.out 462.out 220.out 412.out 132.out 179.out 037.out 293.out 492.out 499.out 112.out 748.out 020.out 786.out 063.out 561.out 076.out 044.out 014.out 362.out 387.out 594.out 438.out 721.out 289.out 574.out 284.out 548.out 299.out 419.out 521.out 581.out 461.out 477.out 307.out 726.out 107.out 007.out 448.out 262.out 380.out 093.out 479.out 547.out 326.out 226.out 345.out 126.out 025.out 121.out 447.out 761.out 580.out 013.out 347.out 275.out 090.out 361.out 564.out 788.out 066.out 455.out 432.out 747.out 558.out 162.out 147.out 288.out 050.out 396.out 576.out 350.out 354.out 298.out 301.out 360.out 543.out 261.out 734.out 045.out 292.out 255.out 729.out 378.out 346.out 048.out 062.out 529.out 231.out 343.out 486.out 223.out 123.out 023.out 459.out 032.out 798.out 146.out 398.out 168.out 427.out 590.out 280.out 470.out 484.out 491.out 767.out 016.out 488.out 731.out 782.out 374.out 285.out 127.out 328.out 260.out 359.out 531.out 376.out 101.out 266.out 331.out 791.out 743.out 556.out 713.out 024.out 513.out 131.out 213.out 730.out 095.out 031.out 565.out 182.out 272.out 530.out 754.out 573.out 454.out 259.out 449.out 327.out 330.out 777.out 493.out 010.out 287.out 230.out 166.out 130.out 219.out 073.out 370.out 135.out 303.out 203.out 003.out 355.out 386.out 560.out 055.out 245.out 243.out 522.out 787.out 745.out 422.out 394.out 159.out 539.out 524.out 122.out 022.out 424.out 779.out 490.out 795.out 353.out 423.out 134.out 154.out 764.out 271.out 518.out 418.out 318.out 077.out 218.out 118.out 757.out 377.out 172.out 034.out 079.out 469.out 054.out 049.out 765.out 021.out 002.out 456.out 180.out 421.out 712.out 403.out 321.out 057.out 512.out 494.out 221.out 312.out 212.out 538.out 256.out 012.out 195.out 283.out 554.out 397.out 185.out 373.out 720.out 481.out 196.out 520.out 242.out 026.out 420.out 188.out 320.out 555.out 120.out 153.out 402.out 302.out 102.out 717.out 517.out 429.out 417.out 317.out 217.out 117.out 475.out 072.out 778.out 097.out 038.out 084.out 369.out 511.out 148.out 411.out 029.out 311.out 368.out 111.out 011.out 257.out 194.out 170.out 133.out 286.out 756.out 175.out 545.out 714.out 559.out 710.out 510.out 413.out 410.out 310.out 210.out 401.out 201.out 001.out 495.out 114.out 753.out 074.out 145.out 797.out 464.out 091.out 081.out 083.out 800.out 064.out 400.out 300.out 200.out 100.out 375.out 541.out 445.out 269.out 546.out 780.out 553.out 763.out 446.out 080.out 070.out 509.out 409.out 309.out 207.out 108.out 009.out 391.out 004.out 557.out 453.out 236.out 566.out 582.out 735.out 383.out 085.out 334.out 504.out 404.out 776.out /var/lib/xfstests/tests/xfs 2023-06-08T03:53:08.813 INFO:teuthology.orchestra.run.smithi079.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-06-08T03:53:08.822 INFO:teuthology.orchestra.run.smithi079.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-06-08T03:53:08.831 INFO:teuthology.orchestra.run.smithi079.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-06-08T03:53:08.839 INFO:teuthology.orchestra.run.smithi079.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-06-08T03:53:08.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T03:53:08.850 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec ls tests/generic) 2023-06-08T03:53:08.868 INFO:teuthology.orchestra.run.smithi079.stdout:001 2023-06-08T03:53:08.868 INFO:teuthology.orchestra.run.smithi079.stdout:001.out 2023-06-08T03:53:08.869 INFO:teuthology.orchestra.run.smithi079.stdout:002 2023-06-08T03:53:08.869 INFO:teuthology.orchestra.run.smithi079.stdout:002.out 2023-06-08T03:53:08.869 INFO:teuthology.orchestra.run.smithi079.stdout:003 2023-06-08T03:53:08.870 INFO:teuthology.orchestra.run.smithi079.stdout:003.out 2023-06-08T03:53:08.870 INFO:teuthology.orchestra.run.smithi079.stdout:004 2023-06-08T03:53:08.870 INFO:teuthology.orchestra.run.smithi079.stdout:004.out 2023-06-08T03:53:08.870 INFO:teuthology.orchestra.run.smithi079.stdout:005 2023-06-08T03:53:08.871 INFO:teuthology.orchestra.run.smithi079.stdout:005.out 2023-06-08T03:53:08.871 INFO:teuthology.orchestra.run.smithi079.stdout:006 2023-06-08T03:53:08.871 INFO:teuthology.orchestra.run.smithi079.stdout:006.out 2023-06-08T03:53:08.871 INFO:teuthology.orchestra.run.smithi079.stdout:007 2023-06-08T03:53:08.872 INFO:teuthology.orchestra.run.smithi079.stdout:007.out 2023-06-08T03:53:08.872 INFO:teuthology.orchestra.run.smithi079.stdout:008 2023-06-08T03:53:08.872 INFO:teuthology.orchestra.run.smithi079.stdout:008.out 2023-06-08T03:53:08.872 INFO:teuthology.orchestra.run.smithi079.stdout:009 2023-06-08T03:53:08.873 INFO:teuthology.orchestra.run.smithi079.stdout:009.out 2023-06-08T03:53:08.873 INFO:teuthology.orchestra.run.smithi079.stdout:010 2023-06-08T03:53:08.873 INFO:teuthology.orchestra.run.smithi079.stdout:010.out 2023-06-08T03:53:08.873 INFO:teuthology.orchestra.run.smithi079.stdout:011 2023-06-08T03:53:08.873 INFO:teuthology.orchestra.run.smithi079.stdout:011.out 2023-06-08T03:53:08.873 INFO:teuthology.orchestra.run.smithi079.stdout:012 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:012.out 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:013 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:013.out 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:014 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:014.out 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:015 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:015.out 2023-06-08T03:53:08.874 INFO:teuthology.orchestra.run.smithi079.stdout:016 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:016.out 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:017 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:017.out 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:018 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:018.out 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:019 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:019.out 2023-06-08T03:53:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:020 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:020.out 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:021 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:021.out 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:022 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:022.out 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:023 2023-06-08T03:53:08.876 INFO:teuthology.orchestra.run.smithi079.stdout:023.out 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:024 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:024.out 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:025 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:025.out 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:026 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:026.out 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:027 2023-06-08T03:53:08.877 INFO:teuthology.orchestra.run.smithi079.stdout:027.out 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:028 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:028.out 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:029 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:029.out 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:030 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:030.out 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:031 2023-06-08T03:53:08.878 INFO:teuthology.orchestra.run.smithi079.stdout:031.out 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:032 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:032.out 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:033 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:033.out 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:034 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:034.out 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:035 2023-06-08T03:53:08.879 INFO:teuthology.orchestra.run.smithi079.stdout:035.cfg 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:035.out.default 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:035.out.nfs 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:036 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:036.out 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:037 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:037.out 2023-06-08T03:53:08.880 INFO:teuthology.orchestra.run.smithi079.stdout:038 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:038.out 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:039 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:039.out 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:040 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:040.out 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:041 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:041.out 2023-06-08T03:53:08.881 INFO:teuthology.orchestra.run.smithi079.stdout:042 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:042.out 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:043 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:043.out 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:044 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:044.out 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:045 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:045.out 2023-06-08T03:53:08.882 INFO:teuthology.orchestra.run.smithi079.stdout:046 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:046.out 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:047 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:047.out 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:048 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:048.out 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:049 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:049.out 2023-06-08T03:53:08.883 INFO:teuthology.orchestra.run.smithi079.stdout:050 2023-06-08T03:53:08.884 INFO:teuthology.orchestra.run.smithi079.stdout:050.cfg 2023-06-08T03:53:08.884 INFO:teuthology.orchestra.run.smithi079.stdout:050.out.default 2023-06-08T03:53:08.884 INFO:teuthology.orchestra.run.smithi079.stdout:050.out.nojournal 2023-06-08T03:53:08.884 INFO:teuthology.orchestra.run.smithi079.stdout:050.out.xfsquota 2023-06-08T03:53:08.885 INFO:teuthology.orchestra.run.smithi079.stdout:051 2023-06-08T03:53:08.885 INFO:teuthology.orchestra.run.smithi079.stdout:051.out 2023-06-08T03:53:08.885 INFO:teuthology.orchestra.run.smithi079.stdout:052 2023-06-08T03:53:08.885 INFO:teuthology.orchestra.run.smithi079.stdout:052.out 2023-06-08T03:53:08.886 INFO:teuthology.orchestra.run.smithi079.stdout:053 2023-06-08T03:53:08.886 INFO:teuthology.orchestra.run.smithi079.stdout:053.out 2023-06-08T03:53:08.886 INFO:teuthology.orchestra.run.smithi079.stdout:054 2023-06-08T03:53:08.886 INFO:teuthology.orchestra.run.smithi079.stdout:054.out 2023-06-08T03:53:08.886 INFO:teuthology.orchestra.run.smithi079.stdout:055 2023-06-08T03:53:08.887 INFO:teuthology.orchestra.run.smithi079.stdout:055.out 2023-06-08T03:53:08.887 INFO:teuthology.orchestra.run.smithi079.stdout:056 2023-06-08T03:53:08.887 INFO:teuthology.orchestra.run.smithi079.stdout:056.out 2023-06-08T03:53:08.887 INFO:teuthology.orchestra.run.smithi079.stdout:057 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:057.out 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:058 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:058.out 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:059 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:059.out 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:060 2023-06-08T03:53:08.888 INFO:teuthology.orchestra.run.smithi079.stdout:060.out 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:061 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:061.out 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:062 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:062.out 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:063 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:063.out 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:064 2023-06-08T03:53:08.889 INFO:teuthology.orchestra.run.smithi079.stdout:064.out 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:065 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:065.out 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:066 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:066.out 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:067 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:067.out 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:068 2023-06-08T03:53:08.890 INFO:teuthology.orchestra.run.smithi079.stdout:068.out 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:069 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:069.out 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:070 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:070.out 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:071 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:071.out 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:072 2023-06-08T03:53:08.891 INFO:teuthology.orchestra.run.smithi079.stdout:072.out 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:073 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:073.out 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:074 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:074.out 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:075 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:075.out 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:076 2023-06-08T03:53:08.892 INFO:teuthology.orchestra.run.smithi079.stdout:076.out 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:077 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:077.out 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:078 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:078.out 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:079 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:079.out 2023-06-08T03:53:08.893 INFO:teuthology.orchestra.run.smithi079.stdout:080 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:080.out 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:081 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:081.out 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:082 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:082.out 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:083 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:083.out 2023-06-08T03:53:08.894 INFO:teuthology.orchestra.run.smithi079.stdout:084 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:084.out 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:085 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:085.out 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:086 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:086.out 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:087 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:087.out 2023-06-08T03:53:08.895 INFO:teuthology.orchestra.run.smithi079.stdout:088 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:088.out 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:089 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:089.out 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:090 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:090.out 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:091 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:091.out 2023-06-08T03:53:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:092 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:092.out 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:093 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:093.out 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:094 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:094.out 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:095 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:095.out 2023-06-08T03:53:08.897 INFO:teuthology.orchestra.run.smithi079.stdout:096 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:096.out 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:097 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:097.out 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:098 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:098.out 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:099 2023-06-08T03:53:08.898 INFO:teuthology.orchestra.run.smithi079.stdout:099.out 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:100 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:100.out 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:101 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:101.out 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:102 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:102.out 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:103 2023-06-08T03:53:08.899 INFO:teuthology.orchestra.run.smithi079.stdout:103.out 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:104 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:104.out 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:105 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:105.out 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:106 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:106.out 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:107 2023-06-08T03:53:08.900 INFO:teuthology.orchestra.run.smithi079.stdout:107.out 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:108 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:108.out 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:109 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:109.out 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:110 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:110.out 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:111 2023-06-08T03:53:08.901 INFO:teuthology.orchestra.run.smithi079.stdout:111.out 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:112 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:112.out 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:113 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:113.out 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:114 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:114.out 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:115 2023-06-08T03:53:08.902 INFO:teuthology.orchestra.run.smithi079.stdout:115.out 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:116 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:116.out 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:117 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:117.out 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:118 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:118.out 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:119 2023-06-08T03:53:08.903 INFO:teuthology.orchestra.run.smithi079.stdout:119.out 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:120 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:120.out 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:121 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:121.out 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:122 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:122.out 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:123 2023-06-08T03:53:08.904 INFO:teuthology.orchestra.run.smithi079.stdout:123.out 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:124 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:124.out 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:125 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:125.out 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:126 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:126.out 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:127 2023-06-08T03:53:08.905 INFO:teuthology.orchestra.run.smithi079.stdout:127.out 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:128 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:128.out 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:129 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:129.out 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:130 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:130.out 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:131 2023-06-08T03:53:08.906 INFO:teuthology.orchestra.run.smithi079.stdout:131.out 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:132 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:132.out 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:133 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:133.out 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:134 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:134.out 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:135 2023-06-08T03:53:08.907 INFO:teuthology.orchestra.run.smithi079.stdout:135.out 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:136 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:136.out 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:137 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:137.out 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:138 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:138.out 2023-06-08T03:53:08.908 INFO:teuthology.orchestra.run.smithi079.stdout:139 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:139.out 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:140 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:140.out 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:141 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:141.out 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:142 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:142.out 2023-06-08T03:53:08.909 INFO:teuthology.orchestra.run.smithi079.stdout:143 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:143.out 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:144 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:144.out 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:145 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:145.out 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:146 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:146.out 2023-06-08T03:53:08.910 INFO:teuthology.orchestra.run.smithi079.stdout:147 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:147.out 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:148 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:148.out 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:149 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:149.out 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:150 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:150.out 2023-06-08T03:53:08.911 INFO:teuthology.orchestra.run.smithi079.stdout:151 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:151.out 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:152 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:152.out 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:153 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:153.out 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:154 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:154.out 2023-06-08T03:53:08.912 INFO:teuthology.orchestra.run.smithi079.stdout:155 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:155.out 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:156 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:156.out 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:157 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:157.out 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:158 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:158.out 2023-06-08T03:53:08.913 INFO:teuthology.orchestra.run.smithi079.stdout:159 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:159.out 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:160 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:160.out 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:161 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:161.out 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:162 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:162.out 2023-06-08T03:53:08.914 INFO:teuthology.orchestra.run.smithi079.stdout:163 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:163.out 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:164 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:164.out 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:165 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:165.out 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:166 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:166.out 2023-06-08T03:53:08.915 INFO:teuthology.orchestra.run.smithi079.stdout:167 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:167.out 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:168 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:168.out 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:169 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:169.out 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:170 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:170.out 2023-06-08T03:53:08.916 INFO:teuthology.orchestra.run.smithi079.stdout:171 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:171.out 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:172 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:172.out 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:173 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:173.out 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:174 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:174.out 2023-06-08T03:53:08.917 INFO:teuthology.orchestra.run.smithi079.stdout:175 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:175.out 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:176 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:176.out 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:177 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:177.out 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:178 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:178.out 2023-06-08T03:53:08.918 INFO:teuthology.orchestra.run.smithi079.stdout:179 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:179.out 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:180 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:180.out 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:181 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:181.out 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:182 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:182.out 2023-06-08T03:53:08.919 INFO:teuthology.orchestra.run.smithi079.stdout:183 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:183.out 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:184 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:184.out 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:185 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:185.out 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:186 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:186.out 2023-06-08T03:53:08.920 INFO:teuthology.orchestra.run.smithi079.stdout:187 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:187.out 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:188 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:188.out 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:189 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:189.out 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:190 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:190.out 2023-06-08T03:53:08.921 INFO:teuthology.orchestra.run.smithi079.stdout:191 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:191.out 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:192 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:192.out 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:193 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:193.out 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:194 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:194.out 2023-06-08T03:53:08.922 INFO:teuthology.orchestra.run.smithi079.stdout:195 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:195.out 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:196 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:196.out 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:197 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:197.out 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:198 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:198.out 2023-06-08T03:53:08.923 INFO:teuthology.orchestra.run.smithi079.stdout:199 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:199.out 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:200 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:200.out 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:201 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:201.out 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:202 2023-06-08T03:53:08.924 INFO:teuthology.orchestra.run.smithi079.stdout:202.out 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:203 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:203.out 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:204 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:204.out 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:205 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:205.out 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:206 2023-06-08T03:53:08.925 INFO:teuthology.orchestra.run.smithi079.stdout:206.out 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:207 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:207.out 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:208 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:208.out 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:209 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:209.out 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:210 2023-06-08T03:53:08.926 INFO:teuthology.orchestra.run.smithi079.stdout:210.out 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:211 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:211.out 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:212 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:212.out 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:213 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:213.out 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:214 2023-06-08T03:53:08.927 INFO:teuthology.orchestra.run.smithi079.stdout:214.out 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:215 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:215.out 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:216 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:216.out 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:217 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:217.out 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:218 2023-06-08T03:53:08.928 INFO:teuthology.orchestra.run.smithi079.stdout:218.out 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:219 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:219.out 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:220 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:220.out 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:221 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:221.out 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:222 2023-06-08T03:53:08.929 INFO:teuthology.orchestra.run.smithi079.stdout:222.out 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:223 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:223.out 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:224 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:224.out 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:225 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:225.out 2023-06-08T03:53:08.930 INFO:teuthology.orchestra.run.smithi079.stdout:226 2023-06-08T03:53:08.931 INFO:teuthology.orchestra.run.smithi079.stdout:226.out 2023-06-08T03:53:08.931 INFO:teuthology.orchestra.run.smithi079.stdout:227 2023-06-08T03:53:08.931 INFO:teuthology.orchestra.run.smithi079.stdout:227.out 2023-06-08T03:53:08.931 INFO:teuthology.orchestra.run.smithi079.stdout:228 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:228.out 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:229 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:229.out 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:230 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:230.out 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:231 2023-06-08T03:53:08.932 INFO:teuthology.orchestra.run.smithi079.stdout:231.out 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:232 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:232.out 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:233 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:233.out 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:234 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:234.out 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:235 2023-06-08T03:53:08.933 INFO:teuthology.orchestra.run.smithi079.stdout:235.out 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:236 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:236.out 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:237 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:237.out 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:238 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:238.out 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:239 2023-06-08T03:53:08.934 INFO:teuthology.orchestra.run.smithi079.stdout:239.out 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:240 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:240.out 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:241 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:241.out 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:242 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:242.out 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:243 2023-06-08T03:53:08.935 INFO:teuthology.orchestra.run.smithi079.stdout:243.out 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:244 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:244.out 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:245 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:245.out 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:246 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:246.out 2023-06-08T03:53:08.936 INFO:teuthology.orchestra.run.smithi079.stdout:247 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:247.out 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:248 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:248.out 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:249 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:249.out 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:250 2023-06-08T03:53:08.937 INFO:teuthology.orchestra.run.smithi079.stdout:250.out 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:251 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:251.out 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:252 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:252.out 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:253 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:253.out 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:254 2023-06-08T03:53:08.938 INFO:teuthology.orchestra.run.smithi079.stdout:254.out 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:255 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:255.out 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:256 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:256.out 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:257 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:257.out 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:258 2023-06-08T03:53:08.939 INFO:teuthology.orchestra.run.smithi079.stdout:258.out 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:259 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:259.out 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:260 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:260.out 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:261 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:261.out 2023-06-08T03:53:08.940 INFO:teuthology.orchestra.run.smithi079.stdout:262 2023-06-08T03:53:08.941 INFO:teuthology.orchestra.run.smithi079.stdout:262.out 2023-06-08T03:53:08.941 INFO:teuthology.orchestra.run.smithi079.stdout:263 2023-06-08T03:53:08.941 INFO:teuthology.orchestra.run.smithi079.stdout:263.out 2023-06-08T03:53:08.942 INFO:teuthology.orchestra.run.smithi079.stdout:264 2023-06-08T03:53:08.942 INFO:teuthology.orchestra.run.smithi079.stdout:264.out 2023-06-08T03:53:08.942 INFO:teuthology.orchestra.run.smithi079.stdout:265 2023-06-08T03:53:08.943 INFO:teuthology.orchestra.run.smithi079.stdout:265.out 2023-06-08T03:53:08.943 INFO:teuthology.orchestra.run.smithi079.stdout:266 2023-06-08T03:53:08.943 INFO:teuthology.orchestra.run.smithi079.stdout:266.out 2023-06-08T03:53:08.943 INFO:teuthology.orchestra.run.smithi079.stdout:267 2023-06-08T03:53:08.944 INFO:teuthology.orchestra.run.smithi079.stdout:267.out 2023-06-08T03:53:08.944 INFO:teuthology.orchestra.run.smithi079.stdout:268 2023-06-08T03:53:08.944 INFO:teuthology.orchestra.run.smithi079.stdout:268.out 2023-06-08T03:53:08.945 INFO:teuthology.orchestra.run.smithi079.stdout:269 2023-06-08T03:53:08.945 INFO:teuthology.orchestra.run.smithi079.stdout:269.out 2023-06-08T03:53:08.945 INFO:teuthology.orchestra.run.smithi079.stdout:270 2023-06-08T03:53:08.946 INFO:teuthology.orchestra.run.smithi079.stdout:270.out 2023-06-08T03:53:08.946 INFO:teuthology.orchestra.run.smithi079.stdout:271 2023-06-08T03:53:08.946 INFO:teuthology.orchestra.run.smithi079.stdout:271.out 2023-06-08T03:53:08.946 INFO:teuthology.orchestra.run.smithi079.stdout:272 2023-06-08T03:53:08.946 INFO:teuthology.orchestra.run.smithi079.stdout:272.out 2023-06-08T03:53:08.946 INFO:teuthology.orchestra.run.smithi079.stdout:273 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:273.out 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:274 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:274.out 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:275 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:275.out 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:276 2023-06-08T03:53:08.947 INFO:teuthology.orchestra.run.smithi079.stdout:276.out 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:277 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:277.out 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:278 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:278.out 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:279 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:279.out 2023-06-08T03:53:08.948 INFO:teuthology.orchestra.run.smithi079.stdout:280 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:280.out 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:281 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:281.out 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:282 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:282.out 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:283 2023-06-08T03:53:08.949 INFO:teuthology.orchestra.run.smithi079.stdout:283.out 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:284 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:284.out 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:285 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:285.out 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:286 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:286.out 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:287 2023-06-08T03:53:08.950 INFO:teuthology.orchestra.run.smithi079.stdout:287.out 2023-06-08T03:53:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:288 2023-06-08T03:53:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:288.out 2023-06-08T03:53:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:289 2023-06-08T03:53:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:289.out 2023-06-08T03:53:08.951 INFO:teuthology.orchestra.run.smithi079.stdout:290 2023-06-08T03:53:08.952 INFO:teuthology.orchestra.run.smithi079.stdout:290.out 2023-06-08T03:53:08.952 INFO:teuthology.orchestra.run.smithi079.stdout:291 2023-06-08T03:53:08.952 INFO:teuthology.orchestra.run.smithi079.stdout:291.out 2023-06-08T03:53:08.953 INFO:teuthology.orchestra.run.smithi079.stdout:292 2023-06-08T03:53:08.953 INFO:teuthology.orchestra.run.smithi079.stdout:292.out 2023-06-08T03:53:08.953 INFO:teuthology.orchestra.run.smithi079.stdout:293 2023-06-08T03:53:08.953 INFO:teuthology.orchestra.run.smithi079.stdout:293.out 2023-06-08T03:53:08.954 INFO:teuthology.orchestra.run.smithi079.stdout:294 2023-06-08T03:53:08.954 INFO:teuthology.orchestra.run.smithi079.stdout:294.out 2023-06-08T03:53:08.954 INFO:teuthology.orchestra.run.smithi079.stdout:295 2023-06-08T03:53:08.954 INFO:teuthology.orchestra.run.smithi079.stdout:295.out 2023-06-08T03:53:08.954 INFO:teuthology.orchestra.run.smithi079.stdout:296 2023-06-08T03:53:08.954 INFO:teuthology.orchestra.run.smithi079.stdout:296.out 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:297 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:297.out 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:298 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:298.out 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:299 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:299.out 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:300 2023-06-08T03:53:08.955 INFO:teuthology.orchestra.run.smithi079.stdout:300.out 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:301 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:301.out 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:302 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:302.out 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:303 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:303.out 2023-06-08T03:53:08.956 INFO:teuthology.orchestra.run.smithi079.stdout:304 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:304.out 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:305 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:305.out 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:306 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:306.out 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:307 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:307.out 2023-06-08T03:53:08.957 INFO:teuthology.orchestra.run.smithi079.stdout:308 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:308.out 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:309 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:309.out 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:310 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:310.out 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:311 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:311.out 2023-06-08T03:53:08.958 INFO:teuthology.orchestra.run.smithi079.stdout:312 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:312.out 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:313 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:313.out 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:314 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:314.out 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:315 2023-06-08T03:53:08.959 INFO:teuthology.orchestra.run.smithi079.stdout:315.out 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:316 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:316.out 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:317 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:317.out 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:318 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:318.out 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:319 2023-06-08T03:53:08.960 INFO:teuthology.orchestra.run.smithi079.stdout:319.out 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:320 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:320.out 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:321 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:321.out 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:322 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:322.out 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:323 2023-06-08T03:53:08.961 INFO:teuthology.orchestra.run.smithi079.stdout:323.out 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:324 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:324.out 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:325 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:325.out 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:326 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:326.out 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:327 2023-06-08T03:53:08.962 INFO:teuthology.orchestra.run.smithi079.stdout:327.out 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:328 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:328.out 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:329 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:329.out 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:330 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:330.out 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:331 2023-06-08T03:53:08.963 INFO:teuthology.orchestra.run.smithi079.stdout:331.out 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:332 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:332.out 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:333 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:333.out 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:334 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:334.out 2023-06-08T03:53:08.964 INFO:teuthology.orchestra.run.smithi079.stdout:335 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:335.out 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:336 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:336.out 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:337 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:337.out 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:338 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:338.out 2023-06-08T03:53:08.965 INFO:teuthology.orchestra.run.smithi079.stdout:339 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:339.out 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:340 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:340.out 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:341 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:341.out 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:342 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:342.out 2023-06-08T03:53:08.966 INFO:teuthology.orchestra.run.smithi079.stdout:343 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:343.out 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:344 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:344.out 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:345 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:345.out 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:346 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:346.out 2023-06-08T03:53:08.967 INFO:teuthology.orchestra.run.smithi079.stdout:347 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:347.out 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:348 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:348.out 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:349 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:349.out 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:350 2023-06-08T03:53:08.968 INFO:teuthology.orchestra.run.smithi079.stdout:350.out 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:351 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:351.out 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:352 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:352.out 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:353 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:353.out 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:354 2023-06-08T03:53:08.969 INFO:teuthology.orchestra.run.smithi079.stdout:354.out 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:355 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:355.out 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:356 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:356.out 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:357 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:357.out 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:358 2023-06-08T03:53:08.970 INFO:teuthology.orchestra.run.smithi079.stdout:358.out 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:359 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:359.out 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:360 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:360.out 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:361 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:361.out 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:362 2023-06-08T03:53:08.971 INFO:teuthology.orchestra.run.smithi079.stdout:362.out 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:363 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:363.out 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:364 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:364.out 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:365 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:365.out 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:366 2023-06-08T03:53:08.972 INFO:teuthology.orchestra.run.smithi079.stdout:366.out 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:367 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:367.out 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:368 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:368.out 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:369 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:369.out 2023-06-08T03:53:08.973 INFO:teuthology.orchestra.run.smithi079.stdout:370 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:370.out 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:371 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:371.out 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:372 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:372.out 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:373 2023-06-08T03:53:08.974 INFO:teuthology.orchestra.run.smithi079.stdout:373.out 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:374 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:374.out 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:375 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:375.out 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:376 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:376.out 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:377 2023-06-08T03:53:08.975 INFO:teuthology.orchestra.run.smithi079.stdout:377.out 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:378 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:378.out 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:379 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:379.out 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:380 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:380.out 2023-06-08T03:53:08.976 INFO:teuthology.orchestra.run.smithi079.stdout:381 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:381.out 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:382 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:382.out 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:383 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:383.out 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:384 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:384.out 2023-06-08T03:53:08.977 INFO:teuthology.orchestra.run.smithi079.stdout:385 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:385.out 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:386 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:386.out 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:387 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:387.out 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:388 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:388.out 2023-06-08T03:53:08.978 INFO:teuthology.orchestra.run.smithi079.stdout:389 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:389.out 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:390 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:390.out 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:391 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:391.out 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:392 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:392.out 2023-06-08T03:53:08.979 INFO:teuthology.orchestra.run.smithi079.stdout:393 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:393.out 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:394 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:394.out 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:395 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:395.out 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:396 2023-06-08T03:53:08.980 INFO:teuthology.orchestra.run.smithi079.stdout:396.out 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:397 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:397.out 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:398 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:398.out 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:399 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:399.out 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:400 2023-06-08T03:53:08.981 INFO:teuthology.orchestra.run.smithi079.stdout:400.out 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:401 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:401.out 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:402 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:402.out 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:403 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:403.out 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:404 2023-06-08T03:53:08.982 INFO:teuthology.orchestra.run.smithi079.stdout:404.out 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:405 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:405.out 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:406 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:406.out 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:407 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:407.out 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:408 2023-06-08T03:53:08.983 INFO:teuthology.orchestra.run.smithi079.stdout:408.out 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:409 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:409.out 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:410 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:410.out 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:411 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:411.out 2023-06-08T03:53:08.984 INFO:teuthology.orchestra.run.smithi079.stdout:412 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:412.out 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:413 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:413.out 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:414 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:414.out 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:415 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:415.out 2023-06-08T03:53:08.985 INFO:teuthology.orchestra.run.smithi079.stdout:416 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:416.out 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:417 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:417.out 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:418 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:418.out 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:419 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:419.out 2023-06-08T03:53:08.986 INFO:teuthology.orchestra.run.smithi079.stdout:420 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:420.out 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:421 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:421.out 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:422 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:422.out 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:423 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:423.out 2023-06-08T03:53:08.987 INFO:teuthology.orchestra.run.smithi079.stdout:424 2023-06-08T03:53:08.988 INFO:teuthology.orchestra.run.smithi079.stdout:424.out 2023-06-08T03:53:08.988 INFO:teuthology.orchestra.run.smithi079.stdout:425 2023-06-08T03:53:08.988 INFO:teuthology.orchestra.run.smithi079.stdout:425.out 2023-06-08T03:53:08.988 INFO:teuthology.orchestra.run.smithi079.stdout:426 2023-06-08T03:53:08.988 INFO:teuthology.orchestra.run.smithi079.stdout:426.out 2023-06-08T03:53:08.988 INFO:teuthology.orchestra.run.smithi079.stdout:427 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:427.out 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:428 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:428.out 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:429 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:429.out 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:430 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:430.out 2023-06-08T03:53:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:431 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:431.out 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:432 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:432.out 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:433 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:433.out 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:434 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:434.out 2023-06-08T03:53:08.990 INFO:teuthology.orchestra.run.smithi079.stdout:435 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:435.out 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:436 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:436.out 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:437 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:437.out 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:438 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:438.out 2023-06-08T03:53:08.991 INFO:teuthology.orchestra.run.smithi079.stdout:439 2023-06-08T03:53:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:439.out 2023-06-08T03:53:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:440 2023-06-08T03:53:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:440.out 2023-06-08T03:53:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:441 2023-06-08T03:53:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:441.out 2023-06-08T03:53:08.992 INFO:teuthology.orchestra.run.smithi079.stdout:442 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:442.out 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:443 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:443.out 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:444 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:444.out 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:445 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:445.out 2023-06-08T03:53:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:446 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:446.out 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:447 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:447.out 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:448 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:448.out 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:449 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:449.out 2023-06-08T03:53:08.994 INFO:teuthology.orchestra.run.smithi079.stdout:450 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:450.out 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:451 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:451.out 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:452 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:452.out 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:453 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:453.out 2023-06-08T03:53:08.995 INFO:teuthology.orchestra.run.smithi079.stdout:454 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:454.out 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:455 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:455.out 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:456 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:456.out 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:457 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:457.out 2023-06-08T03:53:08.996 INFO:teuthology.orchestra.run.smithi079.stdout:458 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:458.out 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:459 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:459.out 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:460 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:460.out 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:461 2023-06-08T03:53:08.997 INFO:teuthology.orchestra.run.smithi079.stdout:461.out 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:462 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:462.out 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:463 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:463.out 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:464 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:464.out 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:465 2023-06-08T03:53:08.998 INFO:teuthology.orchestra.run.smithi079.stdout:465.out 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:466 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:466.out 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:467 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:467.out 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:468 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:468.out 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:469 2023-06-08T03:53:08.999 INFO:teuthology.orchestra.run.smithi079.stdout:469.out 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:470 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:470.out 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:471 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:471.out 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:472 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:472.out 2023-06-08T03:53:09.000 INFO:teuthology.orchestra.run.smithi079.stdout:473 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:473.out 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:474 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:474.out 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:475 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:475.out 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:476 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:476.out 2023-06-08T03:53:09.001 INFO:teuthology.orchestra.run.smithi079.stdout:477 2023-06-08T03:53:09.002 INFO:teuthology.orchestra.run.smithi079.stdout:477.out 2023-06-08T03:53:09.002 INFO:teuthology.orchestra.run.smithi079.stdout:478 2023-06-08T03:53:09.002 INFO:teuthology.orchestra.run.smithi079.stdout:478.out 2023-06-08T03:53:09.002 INFO:teuthology.orchestra.run.smithi079.stdout:479 2023-06-08T03:53:09.002 INFO:teuthology.orchestra.run.smithi079.stdout:479.out 2023-06-08T03:53:09.002 INFO:teuthology.orchestra.run.smithi079.stdout:480 2023-06-08T03:53:09.003 INFO:teuthology.orchestra.run.smithi079.stdout:480.out 2023-06-08T03:53:09.003 INFO:teuthology.orchestra.run.smithi079.stdout:481 2023-06-08T03:53:09.003 INFO:teuthology.orchestra.run.smithi079.stdout:481.out 2023-06-08T03:53:09.003 INFO:teuthology.orchestra.run.smithi079.stdout:482 2023-06-08T03:53:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:482.out 2023-06-08T03:53:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:483 2023-06-08T03:53:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:483.out 2023-06-08T03:53:09.004 INFO:teuthology.orchestra.run.smithi079.stdout:484 2023-06-08T03:53:09.005 INFO:teuthology.orchestra.run.smithi079.stdout:484.out 2023-06-08T03:53:09.005 INFO:teuthology.orchestra.run.smithi079.stdout:485 2023-06-08T03:53:09.005 INFO:teuthology.orchestra.run.smithi079.stdout:485.out 2023-06-08T03:53:09.005 INFO:teuthology.orchestra.run.smithi079.stdout:486 2023-06-08T03:53:09.005 INFO:teuthology.orchestra.run.smithi079.stdout:486.out 2023-06-08T03:53:09.005 INFO:teuthology.orchestra.run.smithi079.stdout:487 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:487.out 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:488 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:488.out 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:489 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:489.out 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:490 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:490.out 2023-06-08T03:53:09.006 INFO:teuthology.orchestra.run.smithi079.stdout:491 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:491.out 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:492 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:492.out 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:493 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:493.out 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:494 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:494.out 2023-06-08T03:53:09.007 INFO:teuthology.orchestra.run.smithi079.stdout:495 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:495.out 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:496 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:496.out 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:497 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:497.out 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:498 2023-06-08T03:53:09.008 INFO:teuthology.orchestra.run.smithi079.stdout:498.out 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:499 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:499.out 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:500 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:500.out 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:501 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:501.out 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:502 2023-06-08T03:53:09.009 INFO:teuthology.orchestra.run.smithi079.stdout:502.out 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:503 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:503.out 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:504 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:504.out 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:505 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:505.out 2023-06-08T03:53:09.010 INFO:teuthology.orchestra.run.smithi079.stdout:506 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:506.out 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:507 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:507.out 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:508 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:508.out 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:509 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:509.out 2023-06-08T03:53:09.011 INFO:teuthology.orchestra.run.smithi079.stdout:510 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:510.out 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:511 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:511.out 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:512 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:512.out 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:513 2023-06-08T03:53:09.012 INFO:teuthology.orchestra.run.smithi079.stdout:513.out 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:514 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:514.out 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:515 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:515.out 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:516 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:516.out 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:517 2023-06-08T03:53:09.013 INFO:teuthology.orchestra.run.smithi079.stdout:517.out 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:518 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:518.out 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:519 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:519.out 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:520 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:520.out 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:521 2023-06-08T03:53:09.014 INFO:teuthology.orchestra.run.smithi079.stdout:521.out 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:522 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:522.out 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:523 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:523.out 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:524 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:524.out 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:525 2023-06-08T03:53:09.015 INFO:teuthology.orchestra.run.smithi079.stdout:525.out 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:526 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:526.out 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:527 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:527.out 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:528 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:528.out 2023-06-08T03:53:09.016 INFO:teuthology.orchestra.run.smithi079.stdout:529 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:529.out 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:530 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:530.out 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:531 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:531.out 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:532 2023-06-08T03:53:09.017 INFO:teuthology.orchestra.run.smithi079.stdout:532.out 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:533 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:533.out 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:534 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:534.out 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:535 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:535.out 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:536 2023-06-08T03:53:09.018 INFO:teuthology.orchestra.run.smithi079.stdout:536.out 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:537 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:537.out 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:538 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:538.out 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:539 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:539.out 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:540 2023-06-08T03:53:09.019 INFO:teuthology.orchestra.run.smithi079.stdout:540.out 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:541 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:541.out 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:542 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:542.out 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:543 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:543.out 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:544 2023-06-08T03:53:09.020 INFO:teuthology.orchestra.run.smithi079.stdout:544.out 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:545 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:545.out 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:546 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:546.out 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:547 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:547.out 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:548 2023-06-08T03:53:09.021 INFO:teuthology.orchestra.run.smithi079.stdout:548.out 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:549 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:549.out 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:550 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:550.out 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:551 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:551.out 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:552 2023-06-08T03:53:09.022 INFO:teuthology.orchestra.run.smithi079.stdout:552.out 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:553 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:553.out 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:554 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:554.out 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:555 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:555.out 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:556 2023-06-08T03:53:09.023 INFO:teuthology.orchestra.run.smithi079.stdout:556.out 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:557 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:557.out 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:558 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:558.out 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:559 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:559.out 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:560 2023-06-08T03:53:09.024 INFO:teuthology.orchestra.run.smithi079.stdout:560.out 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:561 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:561.out 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:562 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:562.out 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:563 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:563.out 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:564 2023-06-08T03:53:09.025 INFO:teuthology.orchestra.run.smithi079.stdout:564.out 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:565 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:565.out 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:566 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:566.out 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:567 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:567.out 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:568 2023-06-08T03:53:09.026 INFO:teuthology.orchestra.run.smithi079.stdout:568.out 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:569 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:569.out 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:570 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:570.out 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:571 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:571.out 2023-06-08T03:53:09.027 INFO:teuthology.orchestra.run.smithi079.stdout:572 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:572.out 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:573 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:573.out 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:574 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:574.out 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:575 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:575.out 2023-06-08T03:53:09.028 INFO:teuthology.orchestra.run.smithi079.stdout:576 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:576.out 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:577 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:577.out 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:578 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:578.out 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:579 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:579.out 2023-06-08T03:53:09.029 INFO:teuthology.orchestra.run.smithi079.stdout:580 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:580.out 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:581 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:581.out 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:582 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:582.out 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:583 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:583.out 2023-06-08T03:53:09.030 INFO:teuthology.orchestra.run.smithi079.stdout:584 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:584.out 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:585 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:585.out 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:586 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:586.out 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:587 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:587.out 2023-06-08T03:53:09.031 INFO:teuthology.orchestra.run.smithi079.stdout:588 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:588.out 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:589 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:589.out 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:590 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:590.out 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:591 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:591.out 2023-06-08T03:53:09.032 INFO:teuthology.orchestra.run.smithi079.stdout:592 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:592.out 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:593 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:593.out 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:594 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:594.out 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:595 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:595.out 2023-06-08T03:53:09.033 INFO:teuthology.orchestra.run.smithi079.stdout:596 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:596.out 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:597 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:597.out 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:598 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:598.out 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:599 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:599.out 2023-06-08T03:53:09.034 INFO:teuthology.orchestra.run.smithi079.stdout:600 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:600.out 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:601 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:601.out 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:602 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:602.out 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:603 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:603.out 2023-06-08T03:53:09.035 INFO:teuthology.orchestra.run.smithi079.stdout:604 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:604.out 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:605 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:605.out 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:606 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:606.out 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:607 2023-06-08T03:53:09.036 INFO:teuthology.orchestra.run.smithi079.stdout:607.out 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:608 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:608.out 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:609 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:609.out 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:610 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:610.out 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:611 2023-06-08T03:53:09.037 INFO:teuthology.orchestra.run.smithi079.stdout:611.out 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:612 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:612.out 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:613 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:613.out 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:614 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:614.out 2023-06-08T03:53:09.038 INFO:teuthology.orchestra.run.smithi079.stdout:615 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:615.out 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:616 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:616.out 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:617 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:617.out 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:618 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:618.out 2023-06-08T03:53:09.039 INFO:teuthology.orchestra.run.smithi079.stdout:619 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:619.out 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:620 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:620.out 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:621 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:621.out 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:622 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:622.out 2023-06-08T03:53:09.040 INFO:teuthology.orchestra.run.smithi079.stdout:623 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:623.out 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:624 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:624.out 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:625 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:625.out 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:626 2023-06-08T03:53:09.041 INFO:teuthology.orchestra.run.smithi079.stdout:626.out 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:627 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:627.out 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:628 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:628.out 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:629 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:629.out 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:630 2023-06-08T03:53:09.042 INFO:teuthology.orchestra.run.smithi079.stdout:630.out 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:631 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:631.out 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:632 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:632.out 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:633 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:633.out 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:634 2023-06-08T03:53:09.043 INFO:teuthology.orchestra.run.smithi079.stdout:634.out 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:635 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:635.out 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:636 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:636.out 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:637 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:637.out 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:638 2023-06-08T03:53:09.044 INFO:teuthology.orchestra.run.smithi079.stdout:638.out 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:639 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:639.out 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:640 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:640.out 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:641 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:641.out 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:642 2023-06-08T03:53:09.045 INFO:teuthology.orchestra.run.smithi079.stdout:642.out 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:643 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:643.out 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:644 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:644.out 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:645 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:645.out 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:646 2023-06-08T03:53:09.046 INFO:teuthology.orchestra.run.smithi079.stdout:646.out 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:647 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:647.out 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:648 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:648.out 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:649 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:649.out 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:650 2023-06-08T03:53:09.047 INFO:teuthology.orchestra.run.smithi079.stdout:650.out 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:651 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:651.out 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:652 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:652.out 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:653 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:653.out 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:654 2023-06-08T03:53:09.048 INFO:teuthology.orchestra.run.smithi079.stdout:654.out 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:655 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:655.out 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:656 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:656.out 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:657 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:657.out 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:658 2023-06-08T03:53:09.049 INFO:teuthology.orchestra.run.smithi079.stdout:658.out 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:659 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:659.out 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:660 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:660.out 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:661 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:661.out 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:662 2023-06-08T03:53:09.050 INFO:teuthology.orchestra.run.smithi079.stdout:662.out 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:663 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:663.out 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:664 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:664.out 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:665 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:665.out 2023-06-08T03:53:09.051 INFO:teuthology.orchestra.run.smithi079.stdout:666 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:666.out 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:667 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:667.out 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:668 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:668.out 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:669 2023-06-08T03:53:09.052 INFO:teuthology.orchestra.run.smithi079.stdout:669.out 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:670 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:670.out 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:671 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:671.out 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:672 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:672.out 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:673 2023-06-08T03:53:09.053 INFO:teuthology.orchestra.run.smithi079.stdout:673.out 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:674 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:674.out 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:675 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:675.out 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:676 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:676.out 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:677 2023-06-08T03:53:09.054 INFO:teuthology.orchestra.run.smithi079.stdout:677.out 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:678 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:678.out 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:679 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:679.out 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:680 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:680.out 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:681 2023-06-08T03:53:09.055 INFO:teuthology.orchestra.run.smithi079.stdout:681.out 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:682 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:682.out 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:683 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:683.out 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:684 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:684.out 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:685 2023-06-08T03:53:09.056 INFO:teuthology.orchestra.run.smithi079.stdout:685.out 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:686 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:686.out 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:687 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:687.out 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:688 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:688.out 2023-06-08T03:53:09.057 INFO:teuthology.orchestra.run.smithi079.stdout:689 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:689.out 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:690 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:690.out 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:691 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:691.out 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:692 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:692.out 2023-06-08T03:53:09.058 INFO:teuthology.orchestra.run.smithi079.stdout:693 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:693.out 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:694 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:694.out 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:695 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:695.out 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:696 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:696.out 2023-06-08T03:53:09.059 INFO:teuthology.orchestra.run.smithi079.stdout:697 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:697.out 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:698 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:698.out 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:699 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:699.out 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:700 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:700.out 2023-06-08T03:53:09.060 INFO:teuthology.orchestra.run.smithi079.stdout:701 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:701.out 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:702 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:702.out 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:703 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:703.out 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:704 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:704.out 2023-06-08T03:53:09.061 INFO:teuthology.orchestra.run.smithi079.stdout:705 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:705.out 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:706 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:706.out 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:707 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:707.out 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:708 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:708.out 2023-06-08T03:53:09.062 INFO:teuthology.orchestra.run.smithi079.stdout:709 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:709.out 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:710 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:710.out 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:711 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:711.out 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:712 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:712.out 2023-06-08T03:53:09.063 INFO:teuthology.orchestra.run.smithi079.stdout:713 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:713.out 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:714 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:714.out 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:715 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:715.out 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:716 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:716.out 2023-06-08T03:53:09.064 INFO:teuthology.orchestra.run.smithi079.stdout:717 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:717.out 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:718 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:718.out 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:719 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:719.out 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:720 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:720.out 2023-06-08T03:53:09.065 INFO:teuthology.orchestra.run.smithi079.stdout:721 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:721.out 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:722 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:722.out 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:723 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:723.out 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:724 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:724.out 2023-06-08T03:53:09.066 INFO:teuthology.orchestra.run.smithi079.stdout:725 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:725.out 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:726 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:726.out 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:727 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:727.out 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:group.list 2023-06-08T03:53:09.067 INFO:teuthology.orchestra.run.smithi079.stdout:Makefile 2023-06-08T03:53:09.068 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-06-08T03:53:09.211 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:09.215 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:09.216 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:11.172 INFO:teuthology.orchestra.run.smithi079.stdout:generic/001 2s 2023-06-08T03:53:11.177 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/001 2023-06-08T03:53:11.177 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:53:11.179 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:11.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi079.stderr:ceph-fuse[118834]: fuse finished with error 0 and tester_r 0 2023-06-08T03:53:11.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:53:11.199 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-06-08T03:53:11.370 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:11.374 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:11.374 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:11.801 INFO:teuthology.orchestra.run.smithi079.stdout:generic/002 0s 2023-06-08T03:53:11.805 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/002 2023-06-08T03:53:11.806 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:53:11.807 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:11.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:53:11.833 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-06-08T03:53:12.022 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:12.026 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:12.026 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:12.147 INFO:teuthology.orchestra.run.smithi079.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T03:53:12.150 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/003 2023-06-08T03:53:12.151 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/003 2023-06-08T03:53:12.151 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:53:12.153 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:12.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:53:12.172 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-06-08T03:53:12.333 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:12.337 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:12.337 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:12.467 INFO:teuthology.orchestra.run.smithi079.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-06-08T03:53:12.471 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/004 2023-06-08T03:53:12.471 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/004 2023-06-08T03:53:12.471 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:53:12.473 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:12.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:53:12.492 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-06-08T03:53:12.655 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:12.659 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:12.659 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:13.177 INFO:teuthology.orchestra.run.smithi079.stdout:generic/005 1s 2023-06-08T03:53:13.182 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/005 2023-06-08T03:53:13.182 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:53:13.184 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:13.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:53:13.208 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-06-08T03:53:13.406 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:13.410 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:13.410 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:32.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:53:32.318 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:53:47.859 INFO:teuthology.orchestra.run.smithi079.stdout:generic/006 34s 2023-06-08T03:53:47.928 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/006 2023-06-08T03:53:47.928 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:53:47.930 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:53:47.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:53:47.954 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-06-08T03:53:48.211 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:53:48.214 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:53:48.215 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:54:02.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:54:02.390 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:54:32.460 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:54:32.462 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:55:01.302 INFO:teuthology.orchestra.run.smithi079.stdout:generic/007 73s 2023-06-08T03:55:01.363 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/007 2023-06-08T03:55:01.364 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:55:01.365 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:01.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:55:01.396 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-06-08T03:55:01.602 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:55:01.606 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:55:01.606 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:02.597 INFO:teuthology.orchestra.run.smithi079.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-06-08T03:55:02.600 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/008 2023-06-08T03:55:02.600 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/008 2023-06-08T03:55:02.600 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:55:02.602 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:02.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:55:02.622 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-06-08T03:55:02.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:55:02.779 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:55:02.841 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:55:02.845 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:55:02.845 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:03.746 INFO:teuthology.orchestra.run.smithi079.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-06-08T03:55:03.750 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/009 2023-06-08T03:55:03.750 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/009 2023-06-08T03:55:03.750 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:55:03.752 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:03.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:55:03.766 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-06-08T03:55:03.928 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:55:03.932 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:55:03.932 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:04.060 INFO:teuthology.orchestra.run.smithi079.stdout:generic/010 [not run] /tmp/tmp.l6an1raz8Jxfstests-dev/src/dbtest not built 2023-06-08T03:55:04.064 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/010 2023-06-08T03:55:04.064 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/010 2023-06-08T03:55:04.064 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:55:04.066 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:04.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:55:04.086 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-06-08T03:55:04.248 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:55:04.252 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:55:04.252 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:33.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:55:33.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:55:44.455 INFO:teuthology.orchestra.run.smithi079.stdout:generic/011 40s 2023-06-08T03:55:44.460 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/011 2023-06-08T03:55:44.460 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:55:44.462 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:44.484 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:55:44.484 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-06-08T03:55:44.684 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:55:44.688 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:55:44.688 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:45.650 INFO:teuthology.orchestra.run.smithi079.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T03:55:45.654 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/012 2023-06-08T03:55:45.654 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/012 2023-06-08T03:55:45.654 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:55:45.656 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:55:45.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:55:45.675 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-06-08T03:55:45.865 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:55:45.870 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:55:45.870 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:56:03.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:56:03.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:56:33.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:56:33.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:57:04.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:57:04.222 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:57:28.967 INFO:teuthology.orchestra.run.smithi079.stdout:generic/013 103s 2023-06-08T03:57:29.039 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/013 2023-06-08T03:57:29.039 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T03:57:29.042 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:57:29.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T03:57:29.053 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-06-08T03:57:29.368 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T03:57:29.373 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T03:57:29.373 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T03:57:34.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:57:34.643 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:58:04.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:58:04.904 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T03:58:04.979 INFO:tasks.ceph.mds.a.smithi079.stderr:2023-06-08T03:58:04.975+0000 7f3370bb3700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 164631) UID: 0 2023-06-08T03:58:04.986 INFO:tasks.ceph.mds.c.smithi079.stderr:2023-06-08T03:58:04.975+0000 7f7ff9adf700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 164631) UID: 0 2023-06-08T04:01:17.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:01:17.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:01:47.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:01:47.253 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:02:08.875 INFO:teuthology.orchestra.run.smithi079.stdout:generic/014 279s 2023-06-08T04:02:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/014 2023-06-08T04:02:08.896 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:08.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:08.922 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-06-08T04:02:09.309 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:09.313 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:09.313 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:09.509 INFO:teuthology.orchestra.run.smithi079.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:02:09.513 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/015 2023-06-08T04:02:09.513 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/015 2023-06-08T04:02:09.513 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:09.516 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:09.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:09.534 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-06-08T04:02:09.800 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:09.807 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:09.807 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:11.463 INFO:teuthology.orchestra.run.smithi079.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:02:11.467 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/016 2023-06-08T04:02:11.468 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/016 2023-06-08T04:02:11.468 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:11.469 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:11.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:11.492 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-06-08T04:02:11.979 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:11.995 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:11.995 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:12.173 INFO:teuthology.orchestra.run.smithi079.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:02:12.177 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/017 2023-06-08T04:02:12.177 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/017 2023-06-08T04:02:12.177 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:12.180 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:12.197 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:12.197 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-06-08T04:02:12.476 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:12.478 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:12.479 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:12.677 INFO:teuthology.orchestra.run.smithi079.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:02:12.681 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/018 2023-06-08T04:02:12.681 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/018 2023-06-08T04:02:12.681 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:12.683 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:12.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:12.704 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-06-08T04:02:12.952 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:12.956 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:12.957 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:13.134 INFO:teuthology.orchestra.run.smithi079.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:02:13.138 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/019 2023-06-08T04:02:13.138 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/019 2023-06-08T04:02:13.139 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:13.141 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:13.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:13.159 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-06-08T04:02:13.446 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:13.456 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:13.456 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:17.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:02:17.769 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:02:28.742 INFO:teuthology.orchestra.run.smithi079.stdout:generic/020 - output mismatch (see /tmp/tmp.l6an1raz8Jxfstests-dev/results//generic/020.out.bad) 2023-06-08T04:02:28.745 INFO:teuthology.orchestra.run.smithi079.stdout: --- tests/generic/020.out 2023-06-08 03:51:57.979794673 +0000 2023-06-08T04:02:28.745 INFO:teuthology.orchestra.run.smithi079.stdout: +++ /tmp/tmp.l6an1raz8Jxfstests-dev/results//generic/020.out.bad 2023-06-08 04:02:28.710618507 +0000 2023-06-08T04:02:28.745 INFO:teuthology.orchestra.run.smithi079.stdout: @@ -39,27 +39,6 @@ 2023-06-08T04:02:28.745 INFO:teuthology.orchestra.run.smithi079.stdout: user.snrub="fish2\012" 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: *** add lots of attributes 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: -*** check 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: - *** MAX_ATTRS attribute(s) 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: -*** remove lots of attributes 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: - *** print attributes 2023-06-08T04:02:28.746 INFO:teuthology.orchestra.run.smithi079.stdout: -# file: 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -user.snrub="fish2\012" 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: - 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -*** really long value 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -0000000 00 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -* 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -ATTRSIZE 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -*** set/get/remove really long names (expect failure) 2023-06-08T04:02:28.747 INFO:teuthology.orchestra.run.smithi079.stdout: -attr_set: Invalid argument 2023-06-08T04:02:28.748 INFO:teuthology.orchestra.run.smithi079.stdout: -Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-06-08T04:02:28.748 INFO:teuthology.orchestra.run.smithi079.stdout: -attr_get: Invalid argument 2023-06-08T04:02:28.748 INFO:teuthology.orchestra.run.smithi079.stdout: -Could not get "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-06-08T04:02:28.748 INFO:teuthology.orchestra.run.smithi079.stdout: -attr_remove: Invalid argument 2023-06-08T04:02:28.748 INFO:teuthology.orchestra.run.smithi079.stdout: -Could not remove "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-06-08T04:02:28.748 INFO:teuthology.orchestra.run.smithi079.stdout: -*** check final 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: - *** print attributes 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: -# file: 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: -user.snrub="fish2\012" 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: - 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: -*** delete 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: +attr_set: No space left on device 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: +Could not set "attribute_2335" for /tmp/tmp.umJjrxL0owtest/attribute_167331 2023-06-08T04:02:28.749 INFO:teuthology.orchestra.run.smithi079.stdout: +!!! failed to add "attribute_2335" 2023-06-08T04:02:28.750 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/020 2023-06-08T04:02:28.750 INFO:teuthology.orchestra.run.smithi079.stdout:Failures: generic/020 2023-06-08T04:02:28.750 INFO:teuthology.orchestra.run.smithi079.stdout:Failed 1 of 1 tests 2023-06-08T04:02:28.751 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:28.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T04:02:28.775 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-06-08T04:02:28.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-06-08T04:02:28.776 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-06-08T04:02:29.278 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:29.282 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:29.283 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:30.889 INFO:teuthology.orchestra.run.smithi079.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:02:30.893 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/021 2023-06-08T04:02:30.893 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/021 2023-06-08T04:02:30.893 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:30.896 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:30.922 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:30.922 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-06-08T04:02:31.412 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:31.417 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:31.417 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:32.994 INFO:teuthology.orchestra.run.smithi079.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:02:32.999 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/022 2023-06-08T04:02:32.999 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/022 2023-06-08T04:02:32.999 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:02:33.001 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:33.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:02:33.024 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-06-08T04:02:33.541 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:02:33.551 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:02:33.552 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:02:47.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:02:47.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:03:17.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:03:17.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:03:47.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:03:48.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:03:52.674 INFO:teuthology.orchestra.run.smithi079.stdout:generic/023 79s 2023-06-08T04:03:52.679 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/023 2023-06-08T04:03:52.679 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:03:52.682 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:52.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:03:52.702 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-06-08T04:03:53.135 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:03:53.139 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:03:53.140 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:53.282 INFO:teuthology.orchestra.run.smithi079.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2023-06-08T04:03:53.285 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/024 2023-06-08T04:03:53.286 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/024 2023-06-08T04:03:53.286 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:03:53.288 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:53.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:03:53.325 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-06-08T04:03:53.741 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:03:53.745 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:03:53.745 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:53.894 INFO:teuthology.orchestra.run.smithi079.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2023-06-08T04:03:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/025 2023-06-08T04:03:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/025 2023-06-08T04:03:53.898 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:03:53.900 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:53.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:03:53.949 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-06-08T04:03:54.360 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:03:54.364 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:03:54.364 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:54.526 INFO:teuthology.orchestra.run.smithi079.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-06-08T04:03:54.530 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/026 2023-06-08T04:03:54.530 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/026 2023-06-08T04:03:54.530 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:03:54.532 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:54.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:03:54.554 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-06-08T04:03:54.716 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:03:54.720 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:03:54.721 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:54.840 INFO:teuthology.orchestra.run.smithi079.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:03:54.844 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/027 2023-06-08T04:03:54.844 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/027 2023-06-08T04:03:54.844 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:03:54.846 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:03:54.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:03:54.870 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-06-08T04:03:55.033 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:03:55.037 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:03:55.037 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:00.172 INFO:teuthology.orchestra.run.smithi079.stdout:generic/028 5s 2023-06-08T04:04:00.177 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/028 2023-06-08T04:04:00.177 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:00.179 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:00.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:00.201 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-06-08T04:04:00.364 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:00.368 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:00.368 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:00.489 INFO:teuthology.orchestra.run.smithi079.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:00.492 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/029 2023-06-08T04:04:00.492 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/029 2023-06-08T04:04:00.493 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:00.494 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:00.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:00.517 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-06-08T04:04:00.680 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:00.684 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:00.684 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:00.806 INFO:teuthology.orchestra.run.smithi079.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:00.809 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/030 2023-06-08T04:04:00.809 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/030 2023-06-08T04:04:00.810 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:00.811 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:00.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:00.833 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-06-08T04:04:00.995 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:00.999 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:00.999 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:01.123 INFO:teuthology.orchestra.run.smithi079.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:01.127 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/031 2023-06-08T04:04:01.127 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/031 2023-06-08T04:04:01.127 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:01.129 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:01.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:01.149 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-06-08T04:04:01.311 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:01.315 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:01.315 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:01.435 INFO:teuthology.orchestra.run.smithi079.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:01.439 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/032 2023-06-08T04:04:01.439 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/032 2023-06-08T04:04:01.439 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:01.441 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:01.471 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:01.472 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-06-08T04:04:01.636 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:01.640 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:01.640 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:01.759 INFO:teuthology.orchestra.run.smithi079.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:01.762 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/033 2023-06-08T04:04:01.762 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/033 2023-06-08T04:04:01.763 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:01.764 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:01.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:01.785 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-06-08T04:04:01.947 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:01.951 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:01.952 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:02.092 INFO:teuthology.orchestra.run.smithi079.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:02.095 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/034 2023-06-08T04:04:02.096 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/034 2023-06-08T04:04:02.096 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:02.098 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:02.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:02.119 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-06-08T04:04:02.286 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:02.290 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:02.470 INFO:teuthology.orchestra.run.smithi079.stdout:generic/035 0s 2023-06-08T04:04:02.474 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/035 2023-06-08T04:04:02.474 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:02.476 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:02.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:02.509 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-06-08T04:04:02.924 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:02.928 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:02.928 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:13.148 INFO:teuthology.orchestra.run.smithi079.stdout:generic/036 11s 2023-06-08T04:04:13.153 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/036 2023-06-08T04:04:13.153 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:13.155 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:13.177 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:13.178 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-06-08T04:04:13.607 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:13.611 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:13.611 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:13.734 INFO:teuthology.orchestra.run.smithi079.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:13.737 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/037 2023-06-08T04:04:13.737 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/037 2023-06-08T04:04:13.738 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:13.739 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:13.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:13.759 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-06-08T04:04:13.922 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:13.926 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:13.927 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:14.046 INFO:teuthology.orchestra.run.smithi079.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:14.049 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/038 2023-06-08T04:04:14.049 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/038 2023-06-08T04:04:14.050 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:14.051 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:14.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:14.072 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-06-08T04:04:14.234 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:14.238 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:14.238 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:14.376 INFO:teuthology.orchestra.run.smithi079.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:14.379 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/039 2023-06-08T04:04:14.379 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/039 2023-06-08T04:04:14.380 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:14.381 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:14.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:14.404 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-06-08T04:04:14.568 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:14.572 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:14.572 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:14.706 INFO:teuthology.orchestra.run.smithi079.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:14.709 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/040 2023-06-08T04:04:14.709 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/040 2023-06-08T04:04:14.710 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:14.711 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:14.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:14.741 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-06-08T04:04:14.904 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:14.908 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:14.908 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.044 INFO:teuthology.orchestra.run.smithi079.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:15.047 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/041 2023-06-08T04:04:15.047 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/041 2023-06-08T04:04:15.048 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:15.049 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:15.076 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-06-08T04:04:15.238 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:15.242 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.364 INFO:teuthology.orchestra.run.smithi079.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:15.368 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/042 2023-06-08T04:04:15.368 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/042 2023-06-08T04:04:15.368 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:15.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:15.391 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-06-08T04:04:15.553 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:15.557 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:15.557 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.686 INFO:teuthology.orchestra.run.smithi079.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:15.689 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/043 2023-06-08T04:04:15.689 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/043 2023-06-08T04:04:15.690 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:15.691 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.706 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:15.707 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-06-08T04:04:15.866 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:15.870 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:15.871 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:15.990 INFO:teuthology.orchestra.run.smithi079.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:15.993 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/044 2023-06-08T04:04:15.993 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/044 2023-06-08T04:04:15.994 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:15.995 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:16.009 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-06-08T04:04:16.172 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:16.176 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:16.176 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.305 INFO:teuthology.orchestra.run.smithi079.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:16.308 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/045 2023-06-08T04:04:16.308 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/045 2023-06-08T04:04:16.309 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:16.310 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:16.330 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-06-08T04:04:16.494 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:16.498 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:16.498 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.625 INFO:teuthology.orchestra.run.smithi079.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:16.628 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/046 2023-06-08T04:04:16.628 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/046 2023-06-08T04:04:16.629 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:16.630 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:16.646 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-06-08T04:04:16.807 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:16.811 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:16.812 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.940 INFO:teuthology.orchestra.run.smithi079.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:16.943 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/047 2023-06-08T04:04:16.943 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/047 2023-06-08T04:04:16.944 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:16.945 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:16.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:16.970 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-06-08T04:04:17.132 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:17.136 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:17.136 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:17.256 INFO:teuthology.orchestra.run.smithi079.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:17.260 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/048 2023-06-08T04:04:17.260 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/048 2023-06-08T04:04:17.260 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:17.262 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:17.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:17.279 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-06-08T04:04:17.443 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:17.447 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:17.447 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:17.576 INFO:teuthology.orchestra.run.smithi079.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:17.579 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/049 2023-06-08T04:04:17.580 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/049 2023-06-08T04:04:17.580 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:17.581 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:17.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:17.606 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-06-08T04:04:17.769 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:17.772 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:17.772 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:17.904 INFO:teuthology.orchestra.run.smithi079.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:17.908 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/050 2023-06-08T04:04:17.908 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/050 2023-06-08T04:04:17.908 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:17.910 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:17.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:17.938 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-06-08T04:04:18.100 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:18.104 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:18.105 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:18.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:04:18.197 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:04:18.234 INFO:teuthology.orchestra.run.smithi079.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:18.237 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/051 2023-06-08T04:04:18.237 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/051 2023-06-08T04:04:18.238 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:18.239 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:18.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:18.258 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-06-08T04:04:18.462 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:18.466 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:18.466 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:18.590 INFO:teuthology.orchestra.run.smithi079.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:18.594 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/052 2023-06-08T04:04:18.594 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/052 2023-06-08T04:04:18.594 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:18.596 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:18.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:18.624 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-06-08T04:04:18.786 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:18.790 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:18.790 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:18.912 INFO:teuthology.orchestra.run.smithi079.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:18.915 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/053 2023-06-08T04:04:18.916 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/053 2023-06-08T04:04:18.916 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:18.917 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:18.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:18.942 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-06-08T04:04:19.106 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:19.110 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:19.110 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:19.234 INFO:teuthology.orchestra.run.smithi079.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:19.237 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/054 2023-06-08T04:04:19.238 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/054 2023-06-08T04:04:19.238 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:19.240 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:19.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:19.262 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-06-08T04:04:19.427 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:19.431 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:19.432 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:19.557 INFO:teuthology.orchestra.run.smithi079.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:19.561 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/055 2023-06-08T04:04:19.561 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/055 2023-06-08T04:04:19.561 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:19.563 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:19.585 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:19.586 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-06-08T04:04:19.748 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:19.753 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:19.753 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:19.889 INFO:teuthology.orchestra.run.smithi079.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:19.893 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/056 2023-06-08T04:04:19.893 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/056 2023-06-08T04:04:19.893 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:19.895 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:19.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:19.919 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-06-08T04:04:20.082 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:20.086 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:20.086 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:20.220 INFO:teuthology.orchestra.run.smithi079.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:20.223 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/057 2023-06-08T04:04:20.224 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/057 2023-06-08T04:04:20.224 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:20.226 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:20.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:20.244 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-06-08T04:04:20.408 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:20.412 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:20.412 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:21.327 INFO:teuthology.orchestra.run.smithi079.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:04:21.330 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/058 2023-06-08T04:04:21.330 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/058 2023-06-08T04:04:21.330 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:21.332 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:21.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:21.366 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-06-08T04:04:21.789 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:21.793 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:21.793 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:21.932 INFO:teuthology.orchestra.run.smithi079.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:21.936 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/059 2023-06-08T04:04:21.936 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/059 2023-06-08T04:04:21.936 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:21.938 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:21.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:21.961 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-06-08T04:04:22.123 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:22.127 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:22.127 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:23.028 INFO:teuthology.orchestra.run.smithi079.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:04:23.031 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/060 2023-06-08T04:04:23.031 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/060 2023-06-08T04:04:23.032 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:23.033 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:23.063 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:23.063 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-06-08T04:04:23.488 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:23.492 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:23.492 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:24.401 INFO:teuthology.orchestra.run.smithi079.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:04:24.404 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/061 2023-06-08T04:04:24.404 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/061 2023-06-08T04:04:24.404 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:24.406 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:24.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:24.461 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-06-08T04:04:24.864 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:24.868 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:24.868 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:24.993 INFO:teuthology.orchestra.run.smithi079.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:24.996 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/062 2023-06-08T04:04:24.997 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/062 2023-06-08T04:04:24.997 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:24.999 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:25.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:25.024 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-06-08T04:04:25.186 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:25.190 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:25.190 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:26.099 INFO:teuthology.orchestra.run.smithi079.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-06-08T04:04:26.103 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/063 2023-06-08T04:04:26.103 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/063 2023-06-08T04:04:26.103 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:26.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:26.137 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-06-08T04:04:26.550 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:26.554 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:26.554 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:26.681 INFO:teuthology.orchestra.run.smithi079.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:26.684 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/064 2023-06-08T04:04:26.684 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/064 2023-06-08T04:04:26.685 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:26.686 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:26.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:26.702 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-06-08T04:04:26.862 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:26.866 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:26.867 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:27.005 INFO:teuthology.orchestra.run.smithi079.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:27.009 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/065 2023-06-08T04:04:27.009 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/065 2023-06-08T04:04:27.009 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:27.011 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:27.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:27.025 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-06-08T04:04:27.187 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:27.191 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:27.192 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:27.328 INFO:teuthology.orchestra.run.smithi079.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:27.331 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/066 2023-06-08T04:04:27.331 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/066 2023-06-08T04:04:27.331 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:27.333 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:27.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:27.350 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-06-08T04:04:27.512 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:27.516 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:27.516 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:27.664 INFO:teuthology.orchestra.run.smithi079.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:27.667 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/067 2023-06-08T04:04:27.668 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/067 2023-06-08T04:04:27.668 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:27.670 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:27.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:27.701 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-06-08T04:04:27.863 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:27.867 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:27.868 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:28.003 INFO:teuthology.orchestra.run.smithi079.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:28.006 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/068 2023-06-08T04:04:28.006 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/068 2023-06-08T04:04:28.007 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:28.008 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:28.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:28.034 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-06-08T04:04:28.197 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:28.204 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:28.204 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:28.330 INFO:teuthology.orchestra.run.smithi079.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:04:28.333 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/069 2023-06-08T04:04:28.333 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/069 2023-06-08T04:04:28.334 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:04:28.335 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:28.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:04:28.352 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-06-08T04:04:28.514 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:04:28.518 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:04:28.518 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:04:48.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:04:48.269 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:05:18.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:05:18.351 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:05:35.107 INFO:teuthology.orchestra.run.smithi079.stdout:generic/070 67s 2023-06-08T04:05:35.111 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/070 2023-06-08T04:05:35.112 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:05:35.114 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:35.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:05:35.138 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-06-08T04:05:35.589 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:05:35.593 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:05:35.593 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:35.714 INFO:teuthology.orchestra.run.smithi079.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:05:35.718 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/071 2023-06-08T04:05:35.718 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/071 2023-06-08T04:05:35.718 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:05:35.720 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:35.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:05:35.738 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-06-08T04:05:35.901 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:05:35.905 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:05:35.906 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:37.388 INFO:teuthology.orchestra.run.smithi079.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-06-08T04:05:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/072 2023-06-08T04:05:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/072 2023-06-08T04:05:37.392 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:05:37.395 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:37.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:05:37.415 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-06-08T04:05:37.689 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:05:37.693 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:05:37.694 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:37.904 INFO:teuthology.orchestra.run.smithi079.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:05:37.910 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/073 2023-06-08T04:05:37.910 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/073 2023-06-08T04:05:37.910 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:05:37.915 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:37.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:05:37.946 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-06-08T04:05:38.188 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:05:38.192 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:05:38.193 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:05:48.407 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:05:48.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:06:18.567 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:06:18.586 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:06:48.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:06:48.929 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:07:19.194 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:07:19.196 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:07:49.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:07:49.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:08:19.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:08:19.372 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:08:50.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:08:50.598 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:09:20.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:09:20.994 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:09:51.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:09:51.470 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:10:22.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:10:22.292 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:10:52.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:10:52.499 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:11:22.586 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:11:22.589 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:11:27.260 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:11:27.256+0000 7f4bb180b700 -1 bdev(0x563468ab2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2023-06-08T04:11:28.990 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:11:28.985+0000 7fbb88b9f700 -1 bdev(0x562131500000 /var/lib/ceph/osd/ceph-7/block) aio_submit retries 1 2023-06-08T04:11:52.874 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:11:52.876 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:12:23.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:12:23.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:12:23.268 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:12:23.264+0000 7f4468930700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 107816) UID: 0 2023-06-08T04:12:23.271 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:12:23.265+0000 7f096fe93700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 107816) UID: 0 2023-06-08T04:12:23.272 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:12:23.264+0000 7ff7b0a1a700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 107816) UID: 0 2023-06-08T04:12:23.273 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:12:23.270+0000 7fbba4fed700 -1 Fail to open '/proc/107816/cmdline' error = (2) No such file or directory 2023-06-08T04:12:23.273 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:12:23.270+0000 7fbba4fed700 -1 received signal: Hangup from (PID: 107816) UID: 0 2023-06-08T04:16:00.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:16:00.193 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:16:00.299 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T04:16:00.294+0000 7f75a36e5700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 197944) UID: 0 2023-06-08T04:16:00.299 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T04:16:00.294+0000 7f8d39efb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 197944) UID: 0 2023-06-08T04:16:00.299 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T04:16:00.294+0000 7faf9780b700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 197944) UID: 0 2023-06-08T04:16:00.300 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:16:00.294+0000 7f4bcfc5d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 197944) UID: 0 2023-06-08T04:16:01.922 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:16:01.918+0000 7ff7b0a1a700 -1 Fail to open '/proc/107988/cmdline' error = (2) No such file or directory 2023-06-08T04:16:01.923 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:16:01.918+0000 7ff7b0a1a700 -1 received signal: Hangup from (PID: 107988) UID: 0 2023-06-08T04:16:01.923 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:16:01.918+0000 7f096fe93700 -1 Fail to open '/proc/107988/cmdline' error = (2) No such file or directory 2023-06-08T04:16:01.923 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:16:01.918+0000 7f096fe93700 -1 received signal: Hangup from (PID: 107988) UID: 0 2023-06-08T04:16:01.924 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:16:01.913+0000 7f4468930700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 107988) UID: 0 2023-06-08T04:16:01.924 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:16:01.918+0000 7fbba4fed700 -1 Fail to open '/proc/107988/cmdline' error = (2) No such file or directory 2023-06-08T04:16:01.924 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:16:01.918+0000 7fbba4fed700 -1 received signal: Hangup from (PID: 107988) UID: 0 2023-06-08T04:17:04.364 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:17:04.360+0000 7f4bb280d700 -1 bdev(0x563468ab2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2023-06-08T04:19:46.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:19:46.658 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:19:46.753 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:19:46.750+0000 7ff7b0a1a700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108058) UID: 0 2023-06-08T04:19:46.754 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:19:46.750+0000 7fbba4fed700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108058) UID: 0 2023-06-08T04:19:46.754 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:19:46.750+0000 7f4468930700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108058) UID: 0 2023-06-08T04:19:46.756 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:19:46.750+0000 7f096fe93700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108058) UID: 0 2023-06-08T04:23:41.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:23:41.668 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:23:42.280 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T04:23:42.277+0000 7f75a36e5700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 198496) UID: 0 2023-06-08T04:23:42.281 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:23:42.277+0000 7f4bcfc5d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 198496) UID: 0 2023-06-08T04:23:42.281 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T04:23:42.278+0000 7faf9780b700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 198496) UID: 0 2023-06-08T04:23:42.281 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T04:23:42.278+0000 7f8d39efb700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 198496) UID: 0 2023-06-08T04:28:16.023 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:28:16.018+0000 7f4bae805700 -1 bdev(0x563468ab2000 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2023-06-08T04:34:30.085 INFO:teuthology.orchestra.run.smithi079.stdout:generic/074 1732s 2023-06-08T04:34:30.095 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/074 2023-06-08T04:34:30.095 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:34:30.101 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:34:30.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:34:30.120 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-06-08T04:34:30.820 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:34:30.824 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:34:30.824 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:34:56.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:34:56.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:34:56.600 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:34:56.597+0000 7f4468930700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108960) UID: 0 2023-06-08T04:34:56.601 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:34:56.597+0000 7fbba4fed700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108960) UID: 0 2023-06-08T04:34:56.601 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:34:56.597+0000 7ff7b0a1a700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108960) UID: 0 2023-06-08T04:34:56.601 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:34:56.597+0000 7f096fe93700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 108960) UID: 0 2023-06-08T04:35:08.099 INFO:teuthology.orchestra.run.smithi079.stdout:generic/075 38s 2023-06-08T04:35:08.174 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/075 2023-06-08T04:35:08.174 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:08.176 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:08.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:08.200 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-06-08T04:35:08.670 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:08.674 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:08.675 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:08.800 INFO:teuthology.orchestra.run.smithi079.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:08.803 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/076 2023-06-08T04:35:08.803 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/076 2023-06-08T04:35:08.803 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:08.805 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:08.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:08.823 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-06-08T04:35:08.989 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:08.993 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:08.993 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:12.076 INFO:teuthology.orchestra.run.smithi079.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:12.079 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/077 2023-06-08T04:35:12.080 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/077 2023-06-08T04:35:12.080 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:12.081 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:12.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:12.102 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-06-08T04:35:12.269 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:12.274 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:12.274 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:12.433 INFO:teuthology.orchestra.run.smithi079.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2023-06-08T04:35:12.436 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/078 2023-06-08T04:35:12.437 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/078 2023-06-08T04:35:12.437 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:12.439 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:12.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:12.464 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-06-08T04:35:12.941 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:12.945 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:12.946 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:13.113 INFO:teuthology.orchestra.run.smithi079.stdout:generic/079 [not run] file system doesn't support chattr +ia 2023-06-08T04:35:13.117 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/079 2023-06-08T04:35:13.117 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/079 2023-06-08T04:35:13.117 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:13.119 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:13.140 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:13.140 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-06-08T04:35:13.304 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:13.308 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:13.308 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:15.485 INFO:teuthology.orchestra.run.smithi079.stdout:generic/080 2s 2023-06-08T04:35:15.490 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/080 2023-06-08T04:35:15.490 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:15.493 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:15.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:15.519 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-06-08T04:35:15.692 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:15.696 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:15.697 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:15.832 INFO:teuthology.orchestra.run.smithi079.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:15.836 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/081 2023-06-08T04:35:15.836 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/081 2023-06-08T04:35:15.836 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:15.838 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:15.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:15.859 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-06-08T04:35:16.024 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:16.028 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:16.028 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:16.165 INFO:teuthology.orchestra.run.smithi079.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:16.169 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/082 2023-06-08T04:35:16.169 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/082 2023-06-08T04:35:16.169 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:16.171 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:16.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:16.195 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-06-08T04:35:16.362 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:16.366 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:16.367 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:16.490 INFO:teuthology.orchestra.run.smithi079.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:16.493 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/083 2023-06-08T04:35:16.493 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/083 2023-06-08T04:35:16.493 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:16.495 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:16.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:16.520 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-06-08T04:35:16.685 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:16.689 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:16.689 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:16.813 INFO:teuthology.orchestra.run.smithi079.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:16.817 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/084 2023-06-08T04:35:16.817 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/084 2023-06-08T04:35:16.817 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:16.819 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:16.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:16.845 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-06-08T04:35:17.008 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:17.013 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:17.013 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:17.178 INFO:teuthology.orchestra.run.smithi079.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2023-06-08T04:35:17.181 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/085 2023-06-08T04:35:17.181 INFO:teuthology.orchestra.run.smithi079.stdout:Not run: generic/085 2023-06-08T04:35:17.181 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:17.183 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:17.206 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:17.206 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-06-08T04:35:17.371 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:17.375 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:17.375 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:21.268 INFO:teuthology.orchestra.run.smithi079.stdout:generic/086 4s 2023-06-08T04:35:21.314 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/086 2023-06-08T04:35:21.315 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:21.317 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:21.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:21.418 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-06-08T04:35:22.486 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:22.490 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:22.490 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:22.851 INFO:teuthology.orchestra.run.smithi079.stdout:generic/087 0s 2023-06-08T04:35:22.856 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/087 2023-06-08T04:35:22.856 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:22.858 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:22.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:22.873 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-06-08T04:35:23.332 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:23.336 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:23.336 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:23.536 INFO:teuthology.orchestra.run.smithi079.stdout:generic/088 0s 2023-06-08T04:35:23.541 INFO:teuthology.orchestra.run.smithi079.stdout:Ran: generic/088 2023-06-08T04:35:23.541 INFO:teuthology.orchestra.run.smithi079.stdout:Passed all 1 tests 2023-06-08T04:35:23.543 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:23.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-06-08T04:35:23.562 DEBUG:teuthology.orchestra.run.smithi079:> (cd /tmp/tmp.l6an1raz8Jxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-06-08T04:35:24.043 INFO:teuthology.orchestra.run.smithi079.stdout:FSTYP -- ceph-fuse 2023-06-08T04:35:24.047 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM -- Linux/x86_64 smithi079 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 2023-06-08T04:35:24.048 INFO:teuthology.orchestra.run.smithi079.stdout: 2023-06-08T04:35:31.765 INFO:tasks.ceph.mds.c.smithi079.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f7ff22d0700 time 2023-06-08T04:35:31.762850+0000 2023-06-08T04:35:31.765 INFO:tasks.ceph.mds.c.smithi079.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-08T04:35:31.791 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.791 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8003b94dbf] 2023-06-08T04:35:31.791 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.791 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.791 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.791 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.792 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: clone() 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr:*** Caught signal (Aborted) ** 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr: in thread 7f7ff22d0700 thread_name:MR_Finisher 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr:2023-06-08T04:35:31.787+0000 7f7ff22d0700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f7ff22d0700 time 2023-06-08T04:35:31.762850+0000 2023-06-08T04:35:31.793 INFO:tasks.ceph.mds.c.smithi079.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8003b94dbf] 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.794 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.795 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.796 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: clone() 2023-06-08T04:35:31.796 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.796 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.796 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8002947ce0] 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: gsignal() 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: abort() 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8003b94e19] 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.797 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.798 INFO:tasks.ceph.mds.c.smithi079.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: 18: clone() 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr:2023-06-08T04:35:31.789+0000 7f7ff22d0700 -1 *** Caught signal (Aborted) ** 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: in thread 7f7ff22d0700 thread_name:MR_Finisher 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8002947ce0] 2023-06-08T04:35:31.799 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: gsignal() 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: abort() 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8003b94e19] 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.800 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.801 INFO:tasks.ceph.mds.c.smithi079.stderr: 18: clone() 2023-06-08T04:35:31.802 INFO:tasks.ceph.mds.c.smithi079.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-08T04:35:31.802 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.808 INFO:tasks.ceph.mds.c.smithi079.stderr: -1> 2023-06-08T04:35:31.787+0000 7f7ff22d0700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f7ff22d0700 time 2023-06-08T04:35:31.762850+0000 2023-06-08T04:35:31.808 INFO:tasks.ceph.mds.c.smithi079.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-08T04:35:31.808 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.808 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8003b94dbf] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.809 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.810 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: clone() 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: 0> 2023-06-08T04:35:31.789+0000 7f7ff22d0700 -1 *** Caught signal (Aborted) ** 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: in thread 7f7ff22d0700 thread_name:MR_Finisher 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8002947ce0] 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: gsignal() 2023-06-08T04:35:31.811 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: abort() 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8003b94e19] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.812 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.813 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.813 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.813 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.813 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.814 INFO:tasks.ceph.mds.c.smithi079.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.814 INFO:tasks.ceph.mds.c.smithi079.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.814 INFO:tasks.ceph.mds.c.smithi079.stderr: 18: clone() 2023-06-08T04:35:31.814 INFO:tasks.ceph.mds.c.smithi079.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-08T04:35:31.814 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.819 INFO:tasks.ceph.mds.c.smithi079.stderr: -1> 2023-06-08T04:35:31.787+0000 7f7ff22d0700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: In function 'void SimpleLock::put_xlock()' thread 7f7ff22d0700 time 2023-06-08T04:35:31.762850+0000 2023-06-08T04:35:31.819 INFO:tasks.ceph.mds.c.smithi079.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-4313-ga02638f8/rpm/el8/BUILD/ceph-18.0.0-4313-ga02638f8/src/mds/SimpleLock.h: 417: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock()) 2023-06-08T04:35:31.819 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.819 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8003b94dbf] 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.820 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.821 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: clone() 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: 0> 2023-06-08T04:35:31.789+0000 7f7ff22d0700 -1 *** Caught signal (Aborted) ** 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: in thread 7f7ff22d0700 thread_name:MR_Finisher 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: ceph version 18.0.0-4313-ga02638f8 (a02638f808b8fe11bccdd01222c0e6e2eb71120b) reef (dev) 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8002947ce0] 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: 2: gsignal() 2023-06-08T04:35:31.822 INFO:tasks.ceph.mds.c.smithi079.stderr: 3: abort() 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8003b94e19] 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f8003b94f85] 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 6: (SimpleLock::put_xlock()+0x332) [0x564f27a5d432] 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 7: (Locker::xlock_finish(std::_Rb_tree_const_iterator const&, MutationImpl*, bool*)+0x8c) [0x564f27a4534c] 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 8: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0x407) [0x564f27a4e187] 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 9: (Locker::drop_non_rdlocks(MutationImpl*, std::set, std::allocator >*)+0x5f) [0x564f27a529ff] 2023-06-08T04:35:31.823 INFO:tasks.ceph.mds.c.smithi079.stderr: 10: (Server::reply_client_request(boost::intrusive_ptr&, boost::intrusive_ptr const&)+0x497) [0x564f278a4bd7] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 11: (Server::respond_to_request(boost::intrusive_ptr&, int)+0x350) [0x564f278a5a80] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 12: (Server::_unlink_local_finish(boost::intrusive_ptr&, CDentry*, CDentry*, unsigned long)+0x2ea) [0x564f278c3c3a] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 13: (MDSContext::complete(int)+0x5f) [0x564f27b8c18f] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 14: (MDSIOContextBase::complete(int)+0x524) [0x564f27b8c914] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 15: (MDSLogContextBase::complete(int)+0x5b) [0x564f27b8cccb] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 16: (Finisher::finisher_thread_entry()+0x18d) [0x7f8003c387cd] 2023-06-08T04:35:31.824 INFO:tasks.ceph.mds.c.smithi079.stderr: 17: /lib64/libpthread.so.0(+0x81cf) [0x7f800293d1cf] 2023-06-08T04:35:31.825 INFO:tasks.ceph.mds.c.smithi079.stderr: 18: clone() 2023-06-08T04:35:31.825 INFO:tasks.ceph.mds.c.smithi079.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-08T04:35:31.825 INFO:tasks.ceph.mds.c.smithi079.stderr: 2023-06-08T04:35:33.555 INFO:tasks.ceph.mds.c.smithi079.stderr:daemon-helper: command crashed with signal 6 2023-06-08T04:35:35.085 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-06-08T04:35:41.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2023-06-08T04:35:48.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2023-06-08T04:35:54.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2023-06-08T04:35:58.336 INFO:tasks.ceph.mds.a.smithi079.stderr:2023-06-08T04:35:58.332+0000 7f336cbab700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-08T04:36:01.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~26s 2023-06-08T04:36:08.125 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~33s 2023-06-08T04:36:14.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~40s 2023-06-08T04:36:21.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~46s 2023-06-08T04:36:27.950 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~53s 2023-06-08T04:36:34.556 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~59s 2023-06-08T04:36:41.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~66s 2023-06-08T04:36:47.771 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~73s 2023-06-08T04:36:54.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~79s 2023-06-08T04:37:00.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~86s 2023-06-08T04:37:07.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~93s 2023-06-08T04:37:14.199 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~99s 2023-06-08T04:37:20.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~106s 2023-06-08T04:37:27.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~112s 2023-06-08T04:37:34.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~119s 2023-06-08T04:37:40.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~126s 2023-06-08T04:37:47.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~132s 2023-06-08T04:37:53.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~139s 2023-06-08T04:38:00.455 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~145s 2023-06-08T04:38:07.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~152s 2023-06-08T04:38:13.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~159s 2023-06-08T04:38:20.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~165s 2023-06-08T04:38:26.887 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~172s 2023-06-08T04:38:33.495 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~178s 2023-06-08T04:38:40.104 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~185s 2023-06-08T04:38:46.710 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~192s 2023-06-08T04:38:53.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~198s 2023-06-08T04:38:59.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~205s 2023-06-08T04:39:06.531 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~211s 2023-06-08T04:39:13.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~218s 2023-06-08T04:39:19.744 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~225s 2023-06-08T04:39:26.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~231s 2023-06-08T04:39:32.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~238s 2023-06-08T04:39:39.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~244s 2023-06-08T04:39:46.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~251s 2023-06-08T04:39:52.777 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~258s 2023-06-08T04:39:59.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~264s 2023-06-08T04:40:05.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~271s 2023-06-08T04:40:12.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~278s 2023-06-08T04:40:19.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~284s 2023-06-08T04:40:25.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~291s 2023-06-08T04:40:32.414 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~297s 2023-06-08T04:40:39.020 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~304s 2023-06-08T04:40:39.021 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-06-08T04:40:39.021 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T04:40:39.021 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T04:40:39.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19355 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 ceph-fuse /tmp/tmp.umJjrxL0owtest fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-06-08T04:40:39.059 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi079.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-06-08T04:40:39.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T04:40:39.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T04:40:39.060 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-08T04:40:39.151 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2023-06-08T04:40:39.152 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T04:40:39.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19355 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 ceph-fuse /tmp/tmp.umJjrxL0owtest fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-06-08T04:40:39.170 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi079.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-06-08T04:40:39.171 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2023-06-08T04:40:39.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T04:40:39.171 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-08T04:40:39.228 INFO:teuthology.orchestra.run.smithi079.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-06-08T04:40:39.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T04:40:39.229 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T04:40:39.230 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T04:40:39.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14018 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-08T04:40:39.258 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.1 is not mounted at ubuntu@smithi156.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1 2023-06-08T04:40:39.258 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T04:40:39.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T04:40:39.258 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-08T04:40:39.322 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-06-08T04:40:39.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T04:40:39.323 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-06-08T04:40:39.323 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-08T04:40:39.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14018 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-08T04:40:39.380 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi156.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-06-08T04:40:39.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-06-08T04:40:39.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-08T04:40:39.381 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-08T04:40:39.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-08T04:40:39.436 INFO:teuthology.orchestra.run.smithi156.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-06-08T04:40:41.043 INFO:tasks.ceph.osd.0:Sent signal 15 2023-06-08T04:40:41.044 INFO:tasks.ceph.osd.1:Sent signal 15 2023-06-08T04:40:41.044 INFO:tasks.ceph.osd.2:Sent signal 15 2023-06-08T04:40:41.044 INFO:tasks.ceph.osd.3:Sent signal 15 2023-06-08T04:40:41.045 INFO:tasks.ceph.osd.4:Sent signal 15 2023-06-08T04:40:41.045 INFO:tasks.ceph.osd.5:Sent signal 15 2023-06-08T04:40:41.045 INFO:tasks.ceph.osd.6:Sent signal 15 2023-06-08T04:40:41.046 INFO:tasks.ceph.osd.7:Sent signal 15 2023-06-08T04:40:41.046 INFO:tasks.ceph.mds.a:Sent signal 15 2023-06-08T04:40:41.046 INFO:tasks.ceph.mds.b:Sent signal 15 2023-06-08T04:40:41.047 INFO:tasks.ceph.mds.d:Sent signal 15 2023-06-08T04:40:41.047 INFO:tasks.ceph.mon.a:Sent signal 15 2023-06-08T04:40:41.047 INFO:tasks.ceph.mon.b:Sent signal 15 2023-06-08T04:40:41.048 INFO:tasks.ceph.mon.c:Sent signal 15 2023-06-08T04:40:41.048 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-06-08T04:40:41.048 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-06-08T04:40:41.049 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T04:40:41.041+0000 7f75a36e5700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 99565) UID: 0 2023-06-08T04:40:41.049 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T04:40:41.041+0000 7f75a36e5700 -1 osd.0 41 *** Got signal Terminated *** 2023-06-08T04:40:41.049 INFO:tasks.ceph.osd.0.smithi079.stderr:2023-06-08T04:40:41.041+0000 7f75a36e5700 -1 osd.0 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.050 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T04:40:41.041+0000 7f8d39efb700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 99567) UID: 0 2023-06-08T04:40:41.050 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T04:40:41.041+0000 7f8d39efb700 -1 osd.1 41 *** Got signal Terminated *** 2023-06-08T04:40:41.050 INFO:tasks.ceph.osd.1.smithi079.stderr:2023-06-08T04:40:41.041+0000 7f8d39efb700 -1 osd.1 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.050 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T04:40:41.041+0000 7faf9780b700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 99571) UID: 0 2023-06-08T04:40:41.051 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T04:40:41.041+0000 7faf9780b700 -1 osd.2 41 *** Got signal Terminated *** 2023-06-08T04:40:41.051 INFO:tasks.ceph.osd.2.smithi079.stderr:2023-06-08T04:40:41.041+0000 7faf9780b700 -1 osd.2 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.051 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:40:41.042+0000 7f4bcfc5d700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 99573) UID: 0 2023-06-08T04:40:41.051 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:40:41.042+0000 7f4bcfc5d700 -1 osd.3 41 *** Got signal Terminated *** 2023-06-08T04:40:41.051 INFO:tasks.ceph.osd.3.smithi079.stderr:2023-06-08T04:40:41.042+0000 7f4bcfc5d700 -1 osd.3 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.052 INFO:tasks.ceph.mds.a.smithi079.stderr:2023-06-08T04:40:41.043+0000 7f3370bb3700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 117236) UID: 0 2023-06-08T04:40:41.052 INFO:tasks.ceph.mds.a.smithi079.stderr:2023-06-08T04:40:41.043+0000 7f3370bb3700 -1 mds.a *** got signal Terminated *** 2023-06-08T04:40:41.052 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-06-08T04:40:41.044+0000 7f9fb9f91700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 99246) UID: 0 2023-06-08T04:40:41.052 INFO:tasks.ceph.mon.a.smithi079.stderr:2023-06-08T04:40:41.044+0000 7f9fb9f91700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-08T04:40:41.053 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:40:41.048+0000 7ff7b0a1a700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 99644) UID: 0 2023-06-08T04:40:41.053 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:40:41.048+0000 7ff7b0a1a700 -1 osd.5 41 *** Got signal Terminated *** 2023-06-08T04:40:41.053 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-06-08T04:40:41.048+0000 7ff7b0a1a700 -1 osd.5 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.053 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:40:41.048+0000 7f096fe93700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 99646) UID: 0 2023-06-08T04:40:41.054 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:40:41.048+0000 7f096fe93700 -1 osd.6 41 *** Got signal Terminated *** 2023-06-08T04:40:41.054 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-06-08T04:40:41.048+0000 7f096fe93700 -1 osd.6 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.054 INFO:tasks.ceph.mon.c.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fd0b223f700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 98992) UID: 0 2023-06-08T04:40:41.054 INFO:tasks.ceph.mon.c.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fd0b223f700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-06-08T04:40:41.055 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fbba4fed700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 99649) UID: 0 2023-06-08T04:40:41.055 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fbba4fed700 -1 osd.7 41 *** Got signal Terminated *** 2023-06-08T04:40:41.055 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fbba4fed700 -1 osd.7 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.055 INFO:tasks.ceph.mds.b.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fe4f80b8700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 106726) UID: 0 2023-06-08T04:40:41.055 INFO:tasks.ceph.mds.b.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fe4f80b8700 -1 mds.b *** got signal Terminated *** 2023-06-08T04:40:41.056 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:40:41.048+0000 7f4468930700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 99641) UID: 0 2023-06-08T04:40:41.056 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:40:41.048+0000 7f4468930700 -1 osd.4 41 *** Got signal Terminated *** 2023-06-08T04:40:41.056 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-06-08T04:40:41.048+0000 7f4468930700 -1 osd.4 41 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-08T04:40:41.056 INFO:tasks.ceph.mds.d.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fd2a4144700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 106727) UID: 0 2023-06-08T04:40:41.057 INFO:tasks.ceph.mds.d.smithi156.stderr:2023-06-08T04:40:41.048+0000 7fd2a4144700 -1 mds.d *** got signal Terminated *** 2023-06-08T04:40:41.057 INFO:tasks.ceph.mon.b.smithi156.stderr:2023-06-08T04:40:41.049+0000 7f1fdaf4f700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 98989) UID: 0 2023-06-08T04:40:41.057 INFO:tasks.ceph.mon.b.smithi156.stderr:2023-06-08T04:40:41.049+0000 7f1fdaf4f700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-06-08T04:40:41.249 INFO:tasks.ceph.mgr.y.smithi079.stderr:daemon-helper: command crashed with signal 15 2023-06-08T04:43:19.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:43:19.586 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:43:49.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:43:49.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:44:19.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:44:19.784 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:44:49.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:44:49.871 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:45:19.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:45:19.999 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:45:50.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:45:50.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:46:20.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:46:20.308 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:46:50.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:46:50.462 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:47:20.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:47:20.789 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:47:50.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:47:50.936 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:48:21.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:48:21.082 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:48:51.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:48:51.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:49:21.596 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:49:21.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:49:51.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:49:51.744 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:50:21.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:50:21.898 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:50:52.042 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:50:52.045 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:51:22.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:51:22.451 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:51:52.599 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:51:52.602 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:52:22.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:52:22.847 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:52:52.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:52:52.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:53:23.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:53:23.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:53:53.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:53:53.492 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:54:23.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:54:23.640 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:54:53.757 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:54:53.761 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:55:23.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:55:23.898 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:55:54.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:55:54.038 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:56:24.186 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:56:24.189 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:56:54.324 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:56:54.327 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:57:24.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:57:24.456 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:57:54.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:57:54.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:58:24.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:58:24.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:58:54.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:58:54.876 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:59:25.003 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:59:25.005 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:59:55.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T04:59:55.143 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:00:25.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:00:25.420 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:00:55.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:00:55.552 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:01:25.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:01:25.706 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:01:55.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:01:55.834 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:02:25.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:02:25.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:02:56.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:02:56.227 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:03:26.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:03:26.363 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:03:56.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:03:56.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:04:26.661 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:04:26.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:04:56.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:04:56.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:05:26.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:05:26.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:05:57.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:05:57.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:06:27.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:06:27.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:06:57.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:06:57.359 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:07:27.494 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:07:27.497 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:07:57.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:07:57.567 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:08:27.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:08:27.689 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:08:57.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:08:57.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:09:28.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:09:28.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:09:58.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:09:58.250 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:10:28.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:10:28.396 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:10:58.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:10:58.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:11:28.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:11:28.679 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:11:58.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:11:58.847 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:12:28.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:12:28.978 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:12:59.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:12:59.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:13:29.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:13:29.187 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:13:59.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:13:59.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:14:29.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:14:29.470 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:14:59.622 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:14:59.624 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:15:29.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:15:29.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:15:59.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:15:59.907 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:16:30.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:16:30.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:17:00.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:17:00.198 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:17:30.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:17:30.361 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:18:00.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:18:00.532 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:18:30.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:18:30.670 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:19:00.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:19:00.823 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:19:31.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:19:31.015 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:20:01.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:20:01.164 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:20:31.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:20:31.310 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:21:01.445 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:21:01.447 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:21:31.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:21:31.855 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:22:01.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:22:01.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:22:32.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:22:32.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:23:02.270 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:23:02.273 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:23:32.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:23:32.670 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:24:02.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:24:02.823 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:24:32.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:24:32.985 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:25:03.103 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:25:03.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:25:33.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:25:33.243 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:26:03.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:26:03.397 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:26:33.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:26:33.531 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:27:03.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:27:03.679 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:27:33.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:27:33.776 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:28:04.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:28:04.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:28:34.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:28:34.176 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:29:04.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:29:04.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:29:34.457 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:29:34.460 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:30:04.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:30:04.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:30:34.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:30:34.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:31:05.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:31:05.097 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:31:35.490 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:31:35.493 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:32:05.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:32:05.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:32:35.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:32:35.767 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:33:06.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:33:06.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:33:36.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:33:36.292 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:34:06.443 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:34:06.446 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:34:36.581 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:34:36.584 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:35:06.727 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:35:06.730 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:35:36.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:35:36.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:36:07.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:36:07.013 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:36:37.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:36:37.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:37:07.241 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:37:07.244 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:37:37.570 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:37:37.573 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:38:07.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:38:07.696 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:38:37.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:38:37.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:39:08.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:39:08.254 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:39:38.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:39:38.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:40:08.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:40:08.554 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:40:38.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:40:38.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:41:08.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:41:08.979 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:41:39.358 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:41:39.361 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:42:09.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:42:09.429 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:42:39.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:42:39.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:43:09.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:43:09.721 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:43:39.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:43:39.858 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:44:10.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:44:10.215 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:44:40.346 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:44:40.349 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:45:10.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:45:10.487 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:45:40.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:45:40.615 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:46:10.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:46:10.753 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:46:41.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:46:41.100 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:47:11.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:47:11.228 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:47:41.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:47:41.375 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:48:11.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:48:11.504 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:48:41.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:48:41.742 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:49:11.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:49:11.815 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:49:41.962 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:49:41.965 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:50:12.083 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:50:12.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:50:42.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:50:42.215 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:51:12.349 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:51:12.352 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:51:42.471 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:51:42.474 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:52:12.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:52:12.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:52:42.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:52:42.781 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:53:12.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:53:12.919 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:53:43.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:53:43.230 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:54:13.366 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:54:13.369 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:54:43.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:54:43.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:55:13.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:55:13.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:55:44.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:55:44.052 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:56:14.136 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:56:14.139 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:56:44.274 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:56:44.276 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:57:14.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:57:14.630 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:57:44.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:57:44.755 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:58:14.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:58:14.894 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:58:45.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:58:45.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:59:15.282 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:59:15.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:59:45.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T05:59:45.422 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:00:15.581 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:00:15.584 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:00:45.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:00:45.713 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:01:16.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:01:16.112 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:01:46.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:01:46.239 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:02:16.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:02:16.418 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:02:46.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:02:46.564 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:03:16.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:03:16.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:03:46.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:03:46.854 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:04:17.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:04:17.243 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:04:47.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:04:47.413 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:05:17.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:05:17.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:05:47.919 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:05:47.922 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:06:18.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:06:18.067 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:06:48.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:06:48.229 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:07:18.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:07:18.374 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:07:48.510 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:07:48.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:08:18.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:08:18.633 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:08:48.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:08:48.779 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:09:18.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:09:18.933 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:09:49.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:09:49.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:10:19.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:10:19.384 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:10:49.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:10:49.538 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:11:19.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:11:19.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:11:49.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:11:49.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:12:20.045 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:12:20.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:12:50.188 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:12:50.191 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:13:20.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:13:20.334 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:13:50.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:13:50.489 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:14:20.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:14:20.649 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:14:50.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:14:50.795 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:15:20.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:15:20.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:15:51.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:15:51.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:16:21.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:16:21.253 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:16:51.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:16:51.395 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:17:21.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:17:21.549 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:17:51.683 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:17:51.686 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:18:21.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:18:21.835 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:18:51.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:18:51.978 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:19:22.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:19:22.141 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:19:52.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:19:52.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:20:22.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:20:22.432 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:20:52.558 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:20:52.561 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:21:22.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:21:22.706 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:21:52.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:21:52.853 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:22:22.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:22:22.990 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:22:53.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:22:53.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:23:23.271 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:23:23.273 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:23:53.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:23:53.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:24:23.662 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:24:23.664 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:24:53.894 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:24:53.896 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:25:24.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:25:24.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:25:54.208 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:25:54.210 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:26:24.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:26:24.514 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:26:54.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:26:54.676 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:27:24.803 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:27:24.805 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:27:54.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:27:54.938 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:28:25.078 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:28:25.080 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:28:55.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:28:55.226 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:29:25.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:29:25.372 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:29:55.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:29:55.515 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:30:25.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:30:25.630 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:30:55.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:30:55.795 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:31:25.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:31:25.931 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:31:56.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:31:56.084 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:32:26.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:32:26.220 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:32:56.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:32:56.375 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:33:26.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:33:26.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:33:56.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:33:56.650 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:34:26.777 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:34:26.780 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:34:56.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:34:56.917 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:35:27.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:35:27.046 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:35:57.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:35:57.183 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:36:27.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:36:27.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:36:57.472 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:36:57.474 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:37:27.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:37:27.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:37:57.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:37:57.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:38:27.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:38:27.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:38:57.989 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:38:57.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:39:28.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:39:28.157 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:39:58.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:39:58.291 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:40:28.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:40:28.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:40:58.637 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:40:58.640 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:41:28.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:41:28.779 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:41:58.901 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:41:58.904 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:42:29.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:42:29.053 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:42:59.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:42:59.183 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:43:29.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:43:29.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:43:59.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:43:59.472 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:44:29.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:44:29.628 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:44:59.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:44:59.766 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:45:29.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:45:29.911 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:46:00.057 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:46:00.060 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:46:30.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:46:30.228 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:47:00.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:47:00.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:47:30.491 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:47:30.494 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:48:00.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:48:00.702 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:48:30.825 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:48:30.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:49:00.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:49:00.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:49:31.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:49:31.145 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:50:01.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:50:01.300 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:50:31.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:50:31.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:51:01.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:51:01.596 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:51:31.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:51:31.839 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:52:01.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:52:01.994 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:52:32.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:52:32.142 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:53:02.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:53:02.271 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:53:32.420 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:53:32.426 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:54:02.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:54:02.581 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:54:32.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:54:32.842 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:55:02.995 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:55:03.018 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:55:33.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:55:33.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:56:03.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:56:03.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:56:33.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:56:33.530 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:57:03.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:57:03.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:57:33.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:57:33.879 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:58:04.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:58:04.038 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:58:34.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:58:34.174 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:59:04.323 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:59:04.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:59:34.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T06:59:34.488 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:00:04.624 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:00:04.627 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:00:34.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:00:34.773 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:01:04.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:01:04.923 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:01:35.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:01:35.050 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:02:05.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:02:05.204 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:02:35.364 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:02:35.367 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:03:05.506 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:03:05.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:03:35.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:03:35.670 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:04:05.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:04:05.855 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:04:35.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:04:36.000 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:05:06.383 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:05:06.386 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:05:36.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:05:36.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:06:06.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:06:06.790 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:06:36.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:06:36.911 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:07:07.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:07:07.063 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:07:37.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:07:37.203 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:08:07.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:08:07.334 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:08:37.492 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:08:37.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:09:07.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:09:07.641 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:09:37.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:09:37.926 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:10:08.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:10:08.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:10:38.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:10:38.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:11:08.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:11:08.390 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:11:38.542 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:11:38.545 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:12:08.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:12:08.697 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:12:38.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:12:38.925 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:13:09.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:13:09.073 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:13:39.210 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:13:39.213 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:14:09.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:14:09.356 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:14:39.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:14:39.510 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:15:09.635 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:15:09.638 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:15:39.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:15:39.767 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:16:09.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:16:09.914 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:16:40.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:16:40.076 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:17:10.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:17:10.222 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:17:40.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:17:40.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:18:10.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:18:10.522 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:18:40.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:18:40.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:19:10.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:19:10.818 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:19:40.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:19:40.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:20:11.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:20:11.123 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:20:41.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:20:41.283 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:21:11.427 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:21:11.430 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:21:41.581 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:21:41.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:22:11.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:22:11.725 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:22:42.041 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:22:42.044 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:23:12.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:23:12.180 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:23:42.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:23:42.340 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:24:12.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:24:12.479 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:24:42.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:24:42.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:25:12.771 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:25:12.774 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:25:42.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:25:42.916 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:26:13.035 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:26:13.037 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:26:43.164 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:26:43.167 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:27:13.290 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:27:13.293 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:27:43.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:27:43.437 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:28:13.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:28:13.590 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:28:43.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:28:43.742 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:29:13.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:29:13.880 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:29:44.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:29:44.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:30:14.122 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:30:14.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:30:44.273 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:30:44.275 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:31:14.419 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:31:14.429 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:31:44.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:31:44.592 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:32:14.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:32:14.737 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:32:44.878 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:32:44.881 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:33:15.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:33:15.030 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:33:45.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:33:45.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:34:15.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:34:15.339 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:34:45.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:34:45.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:35:15.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:35:15.635 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:35:45.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:35:45.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:36:15.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:36:15.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:36:46.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:36:46.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:37:16.195 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:37:16.198 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:37:46.333 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:37:46.336 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:38:16.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:38:16.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:38:46.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:38:46.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:39:16.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:39:16.782 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:39:46.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:39:46.918 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:40:17.059 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:40:17.061 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:40:47.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:40:47.207 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:41:17.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:41:17.385 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:41:47.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:41:47.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:42:17.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:42:17.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:42:47.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:42:47.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:43:17.951 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:43:17.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:43:48.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:43:48.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:44:18.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:44:18.292 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:44:48.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:44:48.444 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:45:18.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:45:18.598 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:45:48.756 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:45:48.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:46:18.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:46:18.894 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:46:48.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:46:49.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:47:19.247 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:47:19.249 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:47:49.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:47:49.385 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:48:19.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:48:19.514 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:48:49.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:48:49.652 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:49:19.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:49:19.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:49:49.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:49:49.929 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:50:20.036 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:50:20.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:50:50.192 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:50:50.195 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:51:20.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:51:20.352 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:51:50.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:51:50.487 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:52:20.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:52:20.657 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:52:50.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:52:50.795 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:53:20.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:53:20.931 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:53:51.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:53:51.067 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:54:21.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:54:21.222 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:54:51.370 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:54:51.373 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:55:21.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:55:21.510 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:55:51.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:55:51.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:56:21.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:56:21.788 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:56:51.898 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:56:51.901 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:57:22.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:57:22.063 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:57:52.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:57:52.405 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:58:22.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:58:22.556 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:58:52.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:58:52.693 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:59:22.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:59:22.822 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:59:53.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T07:59:53.010 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:00:23.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:00:23.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:00:53.318 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:00:53.322 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:01:23.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:01:23.485 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:01:53.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:01:53.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:02:23.778 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:02:23.782 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:02:53.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:02:53.943 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:03:24.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:03:24.100 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:03:54.237 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:03:54.243 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:04:24.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:04:24.424 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:04:54.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:04:54.593 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:05:24.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:05:24.738 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:05:54.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:05:54.868 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:06:25.007 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:06:25.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:06:55.311 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:06:55.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:07:25.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:07:25.473 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:07:55.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:07:55.600 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:08:25.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:08:25.764 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:08:55.895 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:08:55.898 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:09:26.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:09:26.055 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:09:56.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:09:56.200 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:10:26.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:10:26.364 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:10:56.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:10:56.509 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:11:26.626 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:11:26.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:11:56.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:11:56.778 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:12:26.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:12:26.926 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:12:57.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:12:57.059 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:13:27.219 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:13:27.222 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:13:57.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:13:57.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:14:27.500 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:14:27.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:14:57.744 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:14:57.746 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:15:27.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:15:27.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:15:58.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:15:58.009 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:16:28.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:16:28.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:16:58.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:16:58.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:17:28.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:17:28.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:17:58.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:17:58.622 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:18:28.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:18:28.772 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:18:58.913 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:18:58.916 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:19:29.062 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:19:29.065 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:19:59.198 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:19:59.201 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:20:29.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:20:29.344 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:20:59.489 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:20:59.492 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:21:29.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:21:29.662 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:21:59.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:21:59.810 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:22:29.949 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:22:29.953 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:23:00.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:23:00.093 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:23:30.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:23:30.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:24:00.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:24:00.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:24:30.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:24:30.696 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:25:00.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:25:00.833 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:25:30.960 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:25:30.963 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:26:01.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:26:01.092 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:26:31.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:26:31.231 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:27:01.380 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:27:01.382 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:27:31.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:27:31.634 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:28:01.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:28:01.770 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:28:31.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:28:31.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:29:02.086 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:29:02.090 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:29:32.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:29:32.233 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:30:02.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:30:02.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:30:32.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:30:32.654 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:31:02.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:31:02.791 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:31:32.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:31:32.935 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:32:03.095 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:32:03.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:32:33.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:32:33.246 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:33:03.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:33:03.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:33:33.459 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:33:33.462 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:34:03.572 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:34:03.575 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:34:33.693 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:34:33.696 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:35:03.835 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:35:03.838 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:35:33.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:35:33.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:36:04.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:36:04.150 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:36:34.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:36:34.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:37:04.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:37:04.425 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:37:34.732 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:37:34.736 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:38:04.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:38:04.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:38:35.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:38:35.021 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:39:05.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:39:05.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:39:35.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:39:35.306 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:40:05.447 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:40:05.449 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:40:35.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:40:35.589 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:41:05.723 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:41:05.726 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:41:35.903 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:41:35.906 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:42:06.064 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:42:06.066 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:42:36.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:42:36.212 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:43:06.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:43:06.334 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:43:36.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:43:36.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:44:06.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:44:06.630 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:44:36.785 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:44:36.788 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:45:06.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:45:06.942 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:45:37.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:45:37.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:46:07.215 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:46:07.221 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:46:37.367 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:46:37.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:47:07.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:47:07.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:47:37.677 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:47:37.679 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:48:07.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:48:07.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:48:37.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:48:37.987 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:49:08.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:49:08.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:49:38.279 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:49:38.283 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:50:08.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:50:08.443 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:50:38.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:50:38.596 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:51:08.722 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:51:08.725 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:51:38.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:51:38.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:52:09.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:52:09.041 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:52:39.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:52:39.165 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:53:09.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:53:09.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:53:39.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:53:39.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:54:09.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:54:09.674 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:54:39.818 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:54:39.821 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:55:09.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:55:09.976 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:55:40.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:55:40.096 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:56:10.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:56:10.241 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:56:40.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:56:40.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:57:10.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:57:10.559 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:57:40.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:57:40.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:58:10.821 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:58:10.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:58:40.961 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:58:40.964 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:59:11.106 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:59:11.110 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:59:41.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T08:59:41.254 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:00:11.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:00:11.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:00:41.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:00:41.546 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:01:11.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:01:11.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:01:41.959 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:01:41.962 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:02:12.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:02:12.084 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:02:42.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:02:42.242 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:03:12.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:03:12.375 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:03:42.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:03:42.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:04:12.655 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:04:12.658 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:04:42.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:04:42.796 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:05:12.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:05:12.933 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:05:43.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:05:43.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:06:13.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:06:13.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:06:43.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:06:43.372 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:07:13.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:07:13.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:07:43.633 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:07:43.659 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:08:13.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:08:13.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:08:43.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:08:43.945 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:09:14.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:09:14.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:09:44.226 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:09:44.230 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:10:14.379 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:10:14.382 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:10:44.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:10:44.526 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:11:14.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:11:14.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:11:44.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:11:45.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:12:15.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:12:15.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:12:45.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:12:45.562 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:13:15.707 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:13:15.710 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:13:45.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:13:45.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:14:15.996 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:14:16.000 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:14:46.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:14:46.154 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:15:16.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:15:16.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:15:46.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:15:46.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:16:16.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:16:16.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:16:46.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:16:46.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:17:16.880 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:17:16.882 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:17:47.027 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:17:47.030 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:18:17.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:18:17.175 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:18:47.309 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:18:47.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:19:17.444 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:19:17.446 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:19:47.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:19:47.595 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:20:17.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:20:17.747 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:20:47.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:20:47.887 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:21:18.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:21:18.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:21:48.157 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:21:48.160 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:22:18.310 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:22:18.313 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:22:48.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:22:48.444 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:23:18.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:23:18.566 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:23:48.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:23:48.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:24:18.853 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:24:18.856 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:24:49.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:24:49.034 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:25:19.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:25:19.187 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:25:49.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:25:49.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:26:19.533 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:26:19.537 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:26:49.676 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:26:49.680 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:27:19.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:27:19.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:27:49.972 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:27:49.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:28:20.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:28:20.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:28:50.250 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:28:50.253 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:29:20.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:29:20.366 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:29:50.508 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:29:50.510 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:30:20.652 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:30:20.654 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:30:50.836 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:30:50.838 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:31:20.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:31:20.969 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:31:51.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:31:51.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:32:21.254 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:32:21.262 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:32:51.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:32:51.411 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:33:21.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:33:21.538 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:33:51.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:33:51.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:34:21.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:34:21.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:34:51.967 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:34:51.970 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:35:22.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:35:22.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:35:52.264 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:35:52.267 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:36:22.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:36:22.391 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:36:52.541 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:36:52.544 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:37:22.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:37:22.710 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:37:52.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:37:52.872 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:38:23.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:38:23.018 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:38:53.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:38:53.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:39:23.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:39:23.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:39:53.450 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:39:53.454 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:40:23.597 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:40:23.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:40:53.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:40:53.741 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:41:23.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:41:23.882 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:41:54.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:41:54.032 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:42:24.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:42:24.181 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:42:54.344 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:42:54.347 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:43:24.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:43:24.490 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:43:54.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:43:54.652 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:44:24.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:44:24.800 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:44:54.920 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:44:54.923 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:45:25.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:45:25.074 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:45:55.200 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:45:55.203 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:46:25.345 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:46:25.348 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:46:55.509 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:46:55.512 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:47:25.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:47:25.649 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:47:55.782 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:47:55.786 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:48:25.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:48:25.942 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:48:56.092 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:48:56.095 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:49:26.245 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:49:26.249 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:49:56.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:49:56.393 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:50:26.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:50:26.567 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:50:56.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:50:56.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:51:26.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:51:26.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:51:56.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:51:56.996 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:52:27.153 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:52:27.156 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:52:57.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:52:57.301 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:53:27.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:53:27.431 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:53:57.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:53:57.615 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:54:27.762 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:54:27.765 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:54:57.899 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:54:57.901 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:55:28.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:55:28.045 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:55:58.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:55:58.184 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:56:28.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:56:28.323 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:56:58.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:56:58.479 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:57:28.621 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:57:28.624 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:57:58.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:57:58.767 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:58:28.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:58:28.905 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:58:59.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:58:59.064 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:59:29.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:59:29.217 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:59:59.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T09:59:59.368 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:00:29.511 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:00:29.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:00:59.657 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:00:59.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:01:29.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:01:29.821 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:01:59.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:01:59.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:02:30.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:02:30.100 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:03:00.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:03:00.227 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:03:30.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:03:30.539 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:04:00.663 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:04:00.667 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:04:30.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:04:30.810 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:05:00.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:05:00.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:05:31.104 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:05:31.108 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:06:01.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:06:01.226 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:06:31.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:06:31.562 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:07:01.689 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:07:01.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:07:31.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:07:31.846 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:08:01.967 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:08:01.970 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:08:32.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:08:32.124 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:09:02.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:09:02.279 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:09:32.418 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:09:32.421 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:10:02.563 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:10:02.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:10:32.728 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:10:32.731 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:11:02.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:11:02.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:11:32.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:11:32.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:12:03.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:12:03.137 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:12:33.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:12:33.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:13:03.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:13:03.425 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:13:33.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:13:33.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:14:03.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:14:03.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:14:33.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:14:33.857 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:15:04.016 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:15:04.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:15:34.277 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:15:34.279 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:16:04.421 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:16:04.424 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:16:34.560 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:16:34.563 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:17:04.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:17:04.704 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:17:34.834 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:17:34.837 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:18:04.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:18:04.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:18:35.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:18:35.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:19:05.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:19:05.285 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:19:35.428 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:19:35.431 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:20:05.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:20:05.569 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:20:35.711 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:20:35.714 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:21:05.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:21:05.844 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:21:36.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:21:36.021 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:22:06.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:22:06.150 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:22:36.276 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:22:36.281 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:23:06.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:23:06.418 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:23:36.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:23:36.548 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:24:06.715 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:24:06.718 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:24:36.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:24:36.872 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:25:07.015 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:25:07.018 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:25:37.172 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:25:37.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:26:07.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:26:07.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:26:37.469 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:26:37.472 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:27:07.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:27:07.610 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:27:37.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:27:37.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:28:07.873 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:28:07.875 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:28:38.017 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:28:38.020 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:29:08.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:29:08.176 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:29:38.302 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:29:38.305 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:30:08.448 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:30:08.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:30:38.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:30:38.596 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:31:08.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:31:08.734 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:31:38.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:31:38.889 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:32:09.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:32:09.042 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:32:39.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:32:39.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:33:09.315 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:33:09.324 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:33:39.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:33:39.457 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:34:09.598 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:34:09.609 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:34:39.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:34:39.737 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:35:09.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:35:09.868 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:35:40.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:35:40.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:36:10.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:36:10.153 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:36:40.312 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:36:40.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:37:10.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:37:10.651 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:37:40.808 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:37:40.814 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:38:10.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:38:10.990 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:38:41.127 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:38:41.130 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:39:11.289 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:39:11.293 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:39:41.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:39:41.440 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:40:11.587 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:40:11.596 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:40:41.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:40:41.823 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:41:11.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:41:11.951 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:41:42.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:41:42.054 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:42:12.199 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:42:12.201 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:42:42.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:42:42.330 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:43:12.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:43:12.490 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:43:42.623 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:43:42.626 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:44:12.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:44:12.778 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:44:42.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:44:42.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:45:13.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:45:13.075 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:45:43.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:45:43.231 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:46:13.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:46:13.364 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:46:43.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:46:43.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:47:13.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:47:13.668 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:47:43.801 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:47:43.806 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:48:13.966 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:48:13.969 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:48:44.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:48:44.114 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:49:14.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:49:14.245 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:49:44.393 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:49:44.396 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:50:14.547 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:50:14.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:50:44.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:50:44.705 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:51:14.847 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:51:14.850 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:51:44.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:51:44.995 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:52:15.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:52:15.137 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:52:45.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:52:45.289 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:53:15.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:53:15.418 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:53:45.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:53:45.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:54:15.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:54:15.677 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:54:45.806 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:54:45.810 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:55:15.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:55:15.966 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:55:46.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:55:46.123 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:56:16.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:56:16.259 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:56:46.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:56:46.412 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:57:16.546 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:57:16.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:57:46.694 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:57:46.697 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:58:16.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:58:16.833 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:58:46.975 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:58:46.978 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:59:17.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:59:17.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:59:47.267 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T10:59:47.272 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:00:17.397 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:00:17.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:00:47.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:00:47.603 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:01:17.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:01:17.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:01:47.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:01:47.929 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:02:18.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:02:18.075 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:02:48.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:02:48.204 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:03:18.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:03:18.345 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:03:48.484 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:03:48.487 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:04:18.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:04:18.634 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:04:48.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:04:48.754 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:05:18.882 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:05:18.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:05:49.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:05:49.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:06:19.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:06:19.221 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:06:49.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:06:49.389 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:07:19.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:07:19.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:07:49.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:07:49.664 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:08:19.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:08:19.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:08:50.080 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:08:50.085 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:09:20.234 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:09:20.241 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:09:50.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:09:50.380 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:10:20.515 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:10:20.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:10:50.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:10:50.695 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:11:20.856 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:11:20.859 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:11:50.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:11:50.987 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:12:21.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:12:21.140 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:12:51.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:12:51.297 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:13:21.447 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:13:21.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:13:51.619 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:13:51.625 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:14:21.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:14:21.795 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:14:51.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:14:51.949 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:15:22.082 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:15:22.085 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:15:52.235 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:15:52.238 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:16:22.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:16:22.408 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:16:52.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:16:52.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:17:22.680 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:17:22.685 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:17:52.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:17:52.823 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:18:22.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:18:22.975 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:18:53.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:18:53.343 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:19:23.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:19:23.479 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:19:53.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:19:53.560 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:20:23.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:20:23.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:20:53.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:20:53.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:21:24.030 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:21:24.035 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:21:54.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:21:54.188 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:22:24.325 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:22:24.328 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:22:54.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:22:54.504 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:23:24.654 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:23:24.657 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:23:54.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:23:54.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:24:24.930 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:24:24.933 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:24:55.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:24:55.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:25:25.214 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:25:25.217 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:25:55.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:25:55.375 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:26:25.504 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:26:25.507 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:26:55.642 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:26:55.644 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:27:25.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:27:25.793 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:27:55.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:27:55.924 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:28:26.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:28:26.059 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:28:56.211 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:28:56.216 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:29:26.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:29:26.351 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:29:56.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:29:56.510 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:30:26.646 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:30:26.649 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:30:56.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:30:56.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:31:26.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:31:26.940 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:31:57.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:31:57.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:32:27.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:32:27.206 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:32:57.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:32:57.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:33:27.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:33:27.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:33:57.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:33:57.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:34:27.796 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:34:27.800 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:34:57.950 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:34:57.952 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:35:28.135 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:35:28.140 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:35:58.308 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:35:58.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:36:28.446 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:36:28.449 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:36:58.594 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:36:58.597 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:37:28.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:37:28.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:37:58.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:37:58.894 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:38:29.038 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:38:29.041 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:38:59.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:38:59.181 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:39:29.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:39:29.317 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:39:59.452 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:39:59.454 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:40:29.606 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:40:29.608 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:40:59.776 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:40:59.780 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:41:29.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:41:29.924 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:42:00.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:42:00.082 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:42:30.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:42:30.225 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:43:00.359 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:43:00.362 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:43:30.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:43:30.507 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:44:00.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:44:00.654 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:44:30.813 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:44:30.816 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:45:00.942 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:45:00.945 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:45:31.055 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:45:31.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:46:01.201 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:46:01.204 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:46:31.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:46:31.354 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:47:01.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:47:01.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:47:31.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:47:31.635 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:48:01.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:48:01.787 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:48:31.954 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:48:31.957 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:49:02.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:49:02.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:49:32.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:49:32.257 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:50:02.384 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:50:02.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:50:32.531 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:50:32.534 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:51:02.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:51:02.688 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:51:32.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:51:32.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:52:02.968 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:52:02.971 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:52:33.110 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:52:33.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:53:03.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:53:03.257 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:53:33.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:53:33.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:54:03.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:54:03.578 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:54:33.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:54:33.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:55:03.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:55:03.873 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:55:34.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:55:34.025 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:56:04.176 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:56:04.179 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:56:34.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:56:34.324 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:57:04.493 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:57:04.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:57:34.627 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:57:34.629 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:58:04.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:58:04.746 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:58:34.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:58:34.947 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:59:05.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:59:05.105 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:59:35.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T11:59:35.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:00:05.392 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:00:05.395 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:00:35.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:00:35.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:01:05.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:01:05.674 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:01:35.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:01:35.825 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:02:06.085 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:02:06.088 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:02:36.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:02:36.225 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:03:06.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:03:06.363 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:03:36.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:03:36.504 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:04:06.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:04:06.651 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:04:36.814 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:04:36.817 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:05:06.952 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:05:06.954 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:05:37.094 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:05:37.097 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:06:07.255 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:06:07.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:06:37.408 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:06:37.411 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:07:07.568 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:07:07.571 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:07:37.688 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:07:37.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:08:07.818 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:08:07.821 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:08:37.947 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:08:37.950 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:09:08.084 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:09:08.087 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:09:38.239 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:09:38.242 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:10:08.377 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:10:08.379 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:10:38.523 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:10:38.526 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:11:08.724 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:11:08.727 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:11:38.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:11:38.875 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:12:08.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:12:08.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:12:39.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:12:39.134 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:13:09.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:13:09.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:13:39.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:13:39.433 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:14:09.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:14:09.767 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:14:39.905 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:14:39.908 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:15:10.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:15:10.054 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:15:40.197 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:15:40.200 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:16:10.342 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:16:10.346 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:16:40.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:16:40.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:17:10.634 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:17:10.637 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:17:40.772 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:17:40.774 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:18:10.927 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:18:10.930 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:18:41.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:18:41.066 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:19:11.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:19:11.220 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:19:41.338 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:19:41.341 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:20:11.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:20:11.478 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:20:41.607 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:20:41.610 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:21:11.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:21:11.745 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:21:41.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:21:41.866 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:22:11.992 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:22:11.995 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:22:42.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:22:42.140 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:23:12.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:23:12.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:23:42.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:23:42.441 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:24:12.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:24:12.587 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:24:42.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:24:42.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:25:12.851 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:25:12.854 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:25:43.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:25:43.005 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:26:13.158 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:26:13.161 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:26:43.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:26:43.301 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:27:13.441 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:27:13.444 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:27:43.579 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:27:43.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:28:13.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:28:13.711 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:28:43.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:28:43.865 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:29:14.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:29:14.023 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:29:44.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:29:44.157 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:30:14.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:30:14.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:30:44.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:30:44.432 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:31:14.583 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:31:14.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:31:44.742 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:31:44.744 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:32:14.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:32:14.873 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:32:45.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:32:45.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:33:15.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:33:15.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:33:45.303 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:33:45.307 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:34:15.458 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:34:15.461 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:34:45.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:34:45.608 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:35:15.753 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:35:15.755 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:35:45.904 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:35:45.907 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:36:16.058 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:36:16.060 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:36:46.187 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:36:46.190 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:37:16.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:37:16.335 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:37:46.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:37:46.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:38:16.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:38:16.628 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:38:46.745 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:38:46.747 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:39:16.891 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:39:16.893 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:39:47.046 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:39:47.049 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:40:17.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:40:17.185 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:40:47.337 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:40:47.340 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:41:17.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:41:17.485 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:41:47.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:41:47.647 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:42:17.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:42:17.784 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:42:47.934 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:42:47.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:43:18.088 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:43:18.091 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:43:48.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:43:48.234 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:44:18.407 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:44:18.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:44:48.545 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:44:48.548 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:45:18.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:45:18.702 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:45:48.864 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:45:48.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:46:19.011 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:46:19.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:46:49.169 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:46:49.172 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:47:19.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:47:19.253 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:47:49.394 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:47:49.398 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:48:19.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:48:19.534 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:48:49.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:48:49.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:49:19.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:49:19.835 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:49:49.978 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:49:49.982 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:50:20.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:50:20.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:50:50.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:50:50.263 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:51:20.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:51:20.418 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:51:50.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:51:50.555 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:52:20.674 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:52:20.677 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:52:50.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:52:50.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:53:20.957 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:53:20.960 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:53:51.111 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:53:51.113 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:54:21.266 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:54:21.268 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:54:51.399 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:54:51.402 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:55:21.549 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:55:21.552 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:55:51.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:55:51.706 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:56:22.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:56:22.084 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:56:52.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:56:52.256 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:57:22.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:57:22.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:57:52.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:57:52.504 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:58:22.648 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:58:22.651 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:58:52.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:58:52.805 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:59:22.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:59:22.967 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:59:53.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T12:59:53.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:00:23.269 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:00:23.272 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:00:53.405 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:00:53.408 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:01:23.557 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:01:23.560 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:01:53.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:01:53.705 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:02:23.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:02:23.866 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:02:54.050 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:02:54.052 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:03:24.181 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:03:24.184 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:03:54.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:03:54.315 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:04:24.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:04:24.469 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:04:54.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:04:54.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:05:24.748 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:05:24.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:05:54.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:05:54.871 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:06:25.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:06:25.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:06:55.203 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:06:55.206 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:07:25.356 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:07:25.358 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:07:55.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:07:55.503 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:08:25.645 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:08:25.647 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:08:55.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:08:55.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:09:25.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:09:25.920 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:09:56.053 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:09:56.055 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:10:26.180 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:10:26.207 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:10:56.360 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:10:56.362 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:11:26.503 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:11:26.506 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:11:56.643 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:11:56.646 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:12:26.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:12:26.778 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:12:56.902 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:12:56.905 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:13:27.047 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:13:27.050 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:13:57.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:13:57.176 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:14:27.317 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:14:27.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:14:57.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:14:57.454 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:15:27.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:15:27.587 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:15:57.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:15:57.737 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:16:27.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:16:27.899 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:16:58.043 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:16:58.046 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:17:28.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:17:28.186 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:17:58.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:17:58.346 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:18:28.514 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:18:28.517 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:18:58.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:18:58.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:19:28.822 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:19:28.824 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:19:58.991 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:19:58.994 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:20:29.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:20:29.145 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:20:59.260 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:20:59.263 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:21:29.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:21:29.441 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:21:59.574 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:21:59.577 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:22:29.713 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:22:29.716 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:22:59.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:22:59.845 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:23:29.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:23:29.999 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:24:00.125 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:24:00.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:24:30.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:24:30.417 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:25:00.559 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:25:00.561 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:25:30.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:25:30.707 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:26:00.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:26:00.828 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:26:30.971 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:26:30.974 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:27:01.109 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:27:01.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:27:31.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:27:31.415 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:28:01.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:28:01.559 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:28:31.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:28:31.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:29:01.830 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:29:01.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:29:31.955 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:29:31.958 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:30:02.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:30:02.096 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:30:32.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:30:32.241 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:31:02.373 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:31:02.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:31:32.538 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:31:32.541 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:32:02.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:32:02.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:32:32.855 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:32:32.857 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:33:03.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:33:03.003 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:33:33.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:33:33.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:34:03.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:34:03.288 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:34:33.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:34:33.470 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:35:03.618 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:35:03.621 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:35:33.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:35:33.750 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:36:03.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:36:03.895 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:36:34.063 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:36:34.066 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:37:04.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:37:04.233 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:37:34.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:37:34.374 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:38:04.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:38:04.555 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:38:34.697 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:38:34.700 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:39:04.842 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:39:04.845 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:39:34.988 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:39:34.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:40:05.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:40:05.166 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:40:35.313 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:40:35.316 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:41:05.462 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:41:05.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:41:35.625 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:41:35.627 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:42:05.754 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:42:05.756 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:42:35.910 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:42:35.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:43:06.071 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:43:06.074 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:43:36.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:43:36.210 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:44:06.378 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:44:06.381 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:44:36.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:44:36.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:45:06.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:45:06.650 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:45:36.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:45:36.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:46:06.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:46:06.940 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:46:37.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:46:37.109 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:47:07.236 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:47:07.239 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:47:37.361 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:47:37.364 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:48:07.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:48:07.523 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:48:37.658 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:48:37.660 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:49:07.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:49:07.822 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:49:37.940 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:49:37.943 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:50:08.066 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:50:08.069 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:50:38.220 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:50:38.223 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:51:08.363 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:51:08.366 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:51:38.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:51:38.518 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:52:08.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:52:08.647 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:52:38.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:52:38.792 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:53:08.936 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:53:08.939 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:53:39.081 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:53:39.084 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:54:09.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:54:09.223 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:54:39.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:54:39.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:55:09.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:55:09.472 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:55:39.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:55:39.618 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:56:09.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:56:09.738 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:56:39.890 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:56:39.893 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:57:10.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:57:10.032 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:57:40.182 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:57:40.184 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:58:10.335 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:58:10.338 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:58:40.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:58:40.515 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:59:10.660 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:59:10.663 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:59:40.798 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T13:59:40.801 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:00:10.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:00:10.947 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:00:41.073 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:00:41.076 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:01:11.217 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:01:11.220 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:01:41.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:01:41.414 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:02:11.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:02:11.567 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:02:41.708 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:02:41.711 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:03:11.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:03:11.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:03:42.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:03:42.103 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:04:12.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:04:12.226 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:04:42.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:04:42.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:05:12.497 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:05:12.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:05:42.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:05:42.634 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:06:12.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:06:12.790 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:06:42.915 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:06:42.917 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:07:13.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:07:13.064 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:07:43.206 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:07:43.208 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:08:13.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:08:13.358 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:08:43.498 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:08:43.500 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:09:13.610 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:09:13.613 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:09:43.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:09:43.768 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:10:13.918 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:10:13.921 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:10:44.068 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:10:44.071 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:11:14.209 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:11:14.214 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:11:44.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:11:44.350 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:12:14.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:12:14.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:12:44.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:12:44.608 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:13:14.743 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:13:14.746 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:13:44.956 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:13:44.959 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:14:15.093 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:14:15.095 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:14:45.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:14:45.235 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:15:15.367 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:15:15.370 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:15:45.513 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:15:45.516 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:16:15.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:16:15.653 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:16:45.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:16:45.846 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:17:16.004 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:17:16.007 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:17:46.138 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:17:46.141 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:18:16.298 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:18:16.301 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:18:46.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:18:46.442 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:19:16.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:19:16.578 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:19:46.705 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:19:46.708 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:20:16.859 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:20:16.861 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:20:46.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:20:46.992 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:21:17.126 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:21:17.128 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:21:47.295 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:21:47.298 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:22:17.431 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:22:17.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:22:47.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:22:47.587 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:23:17.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:23:17.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:23:47.871 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:23:47.874 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:24:18.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:24:18.028 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:24:48.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:24:48.173 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:25:18.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:25:18.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:25:48.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:25:48.482 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:26:18.650 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:26:18.653 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:26:48.812 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:26:48.815 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:27:18.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:27:18.927 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:27:49.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:27:49.071 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:28:19.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:28:19.227 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:28:49.387 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:28:49.390 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:29:19.543 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:29:19.546 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:29:49.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:29:49.698 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:30:19.832 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:30:19.835 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:30:49.986 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:30:49.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:31:20.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:31:20.127 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:31:50.265 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:31:50.267 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:32:20.407 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:32:20.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:32:50.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:32:50.563 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:33:20.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:33:20.702 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:33:50.861 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:33:50.864 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:34:20.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:34:21.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:34:51.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:34:51.147 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:35:21.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:35:21.294 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:35:51.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:35:51.438 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:36:21.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:36:21.574 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:36:51.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:36:51.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:37:21.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:37:21.875 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:37:52.010 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:37:52.013 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:38:22.140 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:38:22.142 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:38:52.301 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:38:52.304 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:39:22.447 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:39:22.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:39:52.601 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:39:52.604 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:40:22.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:40:22.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:40:52.917 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:40:52.920 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:41:23.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:41:23.075 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:41:53.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:41:53.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:42:23.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:42:23.390 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:42:53.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:42:53.528 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:43:23.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:43:23.669 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:43:53.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:43:53.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:44:23.941 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:44:23.944 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:44:54.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:44:54.102 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:45:24.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:45:24.256 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:45:54.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:45:54.385 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:46:24.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:46:24.531 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:46:54.695 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:46:54.697 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:47:24.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:47:24.834 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:47:54.981 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:47:54.984 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:48:25.120 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:48:25.123 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:48:55.256 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:48:55.259 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:49:25.411 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:49:25.414 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:49:55.553 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:49:55.556 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:50:25.673 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:50:25.675 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:50:55.823 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:50:55.826 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:51:25.965 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:51:25.968 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:51:56.123 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:51:56.126 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:52:26.259 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:52:26.262 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:52:56.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:52:56.417 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:53:26.552 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:53:26.554 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:53:56.706 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:53:56.709 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:54:26.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:54:26.873 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:54:57.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:54:57.025 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:55:27.168 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:55:27.171 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:55:57.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:55:57.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:56:27.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:56:27.437 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:56:57.595 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:56:57.598 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:57:27.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:57:27.739 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:57:57.805 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:57:57.808 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:58:27.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:58:27.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:58:58.013 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:58:58.016 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:59:28.159 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:59:28.162 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:59:58.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T14:59:58.300 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:00:28.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:00:28.436 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:00:58.588 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:00:58.591 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:01:28.709 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:01:28.712 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:01:58.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:01:58.857 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:02:29.009 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:02:29.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:02:59.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:02:59.158 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:03:29.284 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:03:29.287 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:03:59.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:03:59.441 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:04:29.592 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:04:29.595 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:04:59.738 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:04:59.741 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:05:29.883 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:05:29.885 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:06:00.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:06:00.031 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:06:30.175 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:06:30.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:07:00.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:07:00.331 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:07:30.466 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:07:30.469 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:08:00.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:08:00.539 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:08:30.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:08:30.694 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:09:00.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:09:00.840 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:09:30.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:09:30.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:10:01.128 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:10:01.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:10:31.275 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:10:31.278 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:11:01.413 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:11:01.415 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:11:31.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:11:31.535 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:12:01.670 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:12:01.673 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:12:31.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:12:31.827 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:13:01.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:13:01.966 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:13:32.115 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:13:32.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:14:02.253 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:14:02.256 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:14:32.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:14:32.407 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:15:02.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:15:02.552 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:15:32.685 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:15:32.689 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:16:02.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:16:02.855 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:16:32.998 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:16:33.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:17:03.143 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:17:03.146 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:17:33.273 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:17:33.276 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:18:03.415 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:18:03.418 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:18:33.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:18:33.585 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:19:03.718 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:19:03.722 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:19:33.865 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:19:33.869 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:20:03.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:20:03.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:20:34.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:20:34.151 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:21:04.327 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:21:04.331 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:21:34.464 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:21:34.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:22:04.620 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:22:04.630 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:22:34.764 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:22:34.781 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:23:04.935 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:23:04.938 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:23:35.090 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:23:35.098 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:24:05.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:24:05.242 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:24:35.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:24:35.402 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:25:05.539 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:25:05.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:25:35.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:25:35.770 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:26:05.953 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:26:05.964 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:26:36.112 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:26:36.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:27:06.272 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:27:06.286 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:27:36.439 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:27:36.444 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:28:06.571 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:28:06.574 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:28:36.736 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:28:36.739 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:29:06.867 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:29:06.870 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:29:37.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:29:37.024 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:30:07.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:30:07.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:30:37.233 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:30:37.237 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:31:07.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:31:07.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:31:37.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:31:37.633 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:32:07.791 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:32:07.794 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:32:37.921 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:32:37.923 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:33:08.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:33:08.070 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:33:38.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:33:38.207 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:34:08.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:34:08.343 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:34:38.555 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:34:38.558 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:35:08.699 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:35:08.709 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:35:38.845 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:35:38.848 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:36:08.990 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:36:08.993 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:36:39.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:36:39.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:37:09.353 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:37:09.356 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:37:39.488 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:37:39.555 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:38:09.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:38:09.878 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-08T15:38:35.409 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-06-08T15:38:35.410 DEBUG:teuthology.task.console_log:Killing console logger for smithi079 2023-06-08T15:38:35.410 DEBUG:teuthology.task.console_log:Killing console logger for smithi156 2023-06-08T15:38:35.411 DEBUG:teuthology.exit:Finished running handlers